Guide

Run on Kubernetes

The Helm chart runs WrenDNS on any Kubernetes from 1.26, on amd64 and arm64. This guide takes you from one pod to a scaled setup with a Raspberry Pi in the same cluster.

One pod or many

mode: allInOne (the default)mode: scaled
What runsOne StatefulSet pod with a volume for lists, the query log, and users.That pod becomes the controller, plus resolver pods that hold no state and join it by themselves.
Good forA home network: one pod answers tens of thousands of queries a second.Availability: a pod can restart or move while the others keep answering.
Where you manage itThe controller serves the web UI and API, keeps the volume, and publishes the configuration. Resolver pods take its configuration and compiled lists, and ship their query log to it.

Don't raise the replica count of the StatefulSet. Every copy would share one identity, one state.db, and one query log; the second pod refuses to start. Use mode: scaled for more pods.

Install

  1. Get the chart's values, every option commented. Keep only what you change in your own file. The chart's schema rejects a mistyped value with its path.
    helm show values oci://ghcr.io/nbacheller/charts/wrendns --version <version> > values.yaml
  2. Install it with your file (give the DNS Service a fixed address in it: below):
    helm install wrendns oci://ghcr.io/nbacheller/charts/wrendns --version <version> \
      -n wrendns --create-namespace -f my-values.yaml
    kubectl -n wrendns get svc wrendns-dns     # EXTERNAL-IP: hand this out in DHCP
  3. Create your admin: open the web UI and use the setup token from the install notes, or create the first admin from a Secret with auth.bootstrapAdmin.existingSecret (keys username, and password or password-hash).

Every value, with its default, is on the Helm values page. Use a release version, or an edge build that follows main.

Real client addresses

Per-device statistics, groups, and rules need each query's real sender. The chart's DNS Service is a LoadBalancer with externalTrafficPolicy: Local, which keeps it; give it a fixed IP with your load balancer's annotation:

MetalLB

service:
  dns:
    annotations:
      metallb.universe.tf/loadBalancerIPs: 192.168.1.53

Cilium

service:
  dns:
    annotations:
      io.cilium/lb-ipam-ips: 192.168.1.53

kube-vip

service:
  dns:
    annotations:
      kube-vip.io/loadbalancerIPs: 192.168.1.53
  • Or answer on the node itself: hostNetwork: true binds port 53 of the node (no LoadBalancer, and MAC addresses are visible). On Ubuntu nodes, turn off systemd-resolved's stub listener first.
  • If every query seems to come from a few addresses, the web UI says Client IPs appear masked. The usual cause is externalTrafficPolicy: Cluster.

One IP for DNS, the web UI, and the cluster port

With externalTrafficPolicy: Local, two Services can share an IP only if they select the same pods. In mode: scaled the DNS Service also selects the resolver pods, so don't give the API Service the DNS Service's IP with a sharing key: DNS would stop on that address. Let the DNS Service carry the other ports instead:

service:
  dns:
    alsoServe:
      api: true        # the web UI and API on the DNS address (only the controller has them)
      cluster: true    # the cluster port, for a node outside Kubernetes

Or keep the web UI on an ingress: ingress.enabled: true with ingress.host, and add your ingress's network to [api] trusted_proxies under config: so the audit log sees real addresses.

Configuration and secrets

  • Your settings go under config:, written after the chart's own (listeners, data directory, ports): upstreams, lists, groups, sign-in, everything in the configuration reference.
  • With Argo CD or Flux, add [cluster] config_source = "gitops". Changes in the web UI then show the TOML to commit: see Git, per node.
  • Secrets as files: secretMounts mounts Secrets at /etc/wrendns-secrets/<name>, for settings that take a file, such as an OIDC client_secret_file or a router's api_key_file.
  • Routers: a resolver pod reads no router (it has no router credentials). The controller, or the primary for a router whose credential is stored on the node (Devices → Routers), reads them and shares the names with every node.
  • With Argo CD, which can't keep a generated Secret stable, create the resolver pods' join Secret yourself and set cluster.bootstrapSecret.existingSecret (at least 32 random characters).
  • Stored secrets entered in the web UI are copied to the controller and to every full member (a Pi, say), each encrypted under its own key. Resolver pods get none: an integration that needs a secret on a pod uses a file (secretMounts) or runs on a full member. The Cluster page's Secrets column shows whether each member has them.
  • The key of stored secrets (those entered in the web UI) is, by default, secrets.key on the controller's volume. Better, keep your own: secretsKey.existingSecret names a Secret whose entry secretsKey.key (default secrets.key) holds ttsk1: and 43 base64url characters; it is mounted read-only on the controller only, never on resolver pods. secretsKey.generate: true makes that Secret with the chart, which doesn't work under helm template or Argo CD (each render would make a new key and lock the store). With persistence.enabled: false stored secrets are lost at every restart, and health says secrets_ephemeral. See docs/secrets.md.
secretMounts:
  - name: oidc
    secretName: wrendns-oidc
config: |
  [cluster]
  config_source = "gitops"

  [api]
  trusted_proxies = ["10.0.0.0/8"]

  [auth.oidc]
  public_url = "https://dns.example.com"

  [[auth.oidc.provider]]
  id = "authentik"
  issuer = "https://auth.example.com/application/o/wrendns/"
  client_id = "..."
  client_secret_file = "/etc/wrendns-secrets/oidc/client_secret"

More pods

mode: scaled
resolvers:
  replicas: 3
  • Spread them: with externalTrafficPolicy: Local, each Kubernetes node sends queries only to its own pods. Use resolvers.affinity (podAntiAffinity on app.kubernetes.io/component: resolver) to put them on different nodes. The Cluster page's Load balance check flags a pod taking more than twice its share.
  • Each pod has its own cache and rate limits. A new pod starts cold, and a client spread over three pods can send three times the per-client limit.
  • Rollouts don't lose lookups: on shutdown a pod reports not ready, then keeps answering for drainDelaySeconds (5) while the Service moves away. A pod that shuts down leaves the cluster's node list at once; one that crashes expires after resolvers.ephemeralTtlSeconds.
  • Resolver pods are trusted less than the controller: they never get the cluster's signing key, can't be promoted, and can't change anything on other nodes. Make changes on the controller.
  • One Kubernetes node is still one machine. For protection against that node going down, run pods on several nodes, or add a Pi as a second DNS server (next).

A Pi in the same cluster

A node outside Kubernetes joins the controller like any cluster member, and makes a good second DNS server for DHCP.

  1. Expose the cluster port (9443 in the chart): service.dns.alsoServe.cluster: true, or cluster.service.type: LoadBalancer. List that address in cluster.advertise, e.g. ["https://192.168.1.53:9443"].
  2. Make a join token inside the controller pod, naming that URL:
    kubectl -n wrendns exec sts/wrendns -- wrendns cluster token create --url https://192.168.1.53:9443 \
      -c /etc/wrendns/00-chart.toml -c /etc/wrendns/10-values.toml -c /etc/wrendns/20-controller.toml
  3. Join from the Pi, then restart it: wrendns cluster join tt_join_... --site home-pi.
  4. Optional: staged rollouts. cluster.rollout.canaries: [ephemeral] gives each change to the resolver pods (and the controller) first, and to the Pi after a five-minute bake, pinning the cluster back if the pods start failing.

The Pi keeps its own [cache], listeners, and sign-in settings in its own file: see Shared or per node.

Your pods as devices

WrenDNS can name your cluster's pods by the workload that runs them, like any laptop or TV on the network: media/deploy/jellyfin for the Deployment jellyfin in media. A Deployment's replicas are one device across every rollout, so its top domains, anomaly baseline, rhythm, and day on the timeline carry on when its pods are replaced. Namespaces get groups, so lists, quick rules, schedules, and blocked services apply per namespace. See what it looks like.

Pods ask WrenDNS directly; it names each one by its workload, answers or forwards, and sends the cluster's own names on to CoreDNS. The controller watches pods (read-only) to know who is who. Pods (their own addresses) jellyfin-7d9f4c8b6-x2k1q10.42.0.15 jellyfin-7d9f4c8b6-m8pqz10.42.1.22 postgres-010.42.0.21 WrenDNS media/deploy/jellyfin: one device db/sts/postgres: one device each in its namespace's group Upstreamsthe internet, filtered per group CoreDNScluster.local and reverse zones Kubernetes API: watch pods and namespaces (read-only)
Pods ask WrenDNS directly, so their own addresses arrive; Service names still come from CoreDNS.
  1. Turn it on with CoreDNS's address (kubectl -n kube-system get svc kube-dns -o jsonpath='{.spec.clusterIP}') and a fixed address for the DNS Service inside the Service network:
    kubernetes:
      enabled: true
      clusterDns: 10.43.0.10
      podCidr: 10.42.0.0/16       # optional: PTR lookups for pods and Services go to CoreDNS too
      serviceCidr: 10.43.0.0/16   #   (leave both out if your nodes use WrenDNS: see below)
    service:
      dns:
        clusterIP: 10.43.0.53
    The controller gets a token that can only read pods and namespaces; resolver pods get none.
  2. Send the pods' DNS to WrenDNS: one workload first, then the whole cluster through the kubelet. The next section walks through it.
  3. Group them. Label a namespace (kubectl label namespace media wrendns.com/group=media) or a pod the same way, list rules in kubernetes.map ([{namespace: "media*", group: "iot"}]), or use Add to group… on a workload in the web UI. Labels win, then rules, then kubernetes.defaultGroup, then the pod's network group.

Through CoreDNS, every pod looks like CoreDNS. If CoreDNS forwards to WrenDNS instead, all you see is CoreDNS's address. The API server being unreachable never affects answering: pods keep their last names, and the health level turns degraded after 15 minutes.

Your cluster's DNS through WrenDNS

Pods only show up as themselves when they ask WrenDNS directly. The safe order is one workload, then the cluster's default, then the nodes. The examples use k3s, with WrenDNS's DNS Service at 10.43.0.53 and CoreDNS at 10.43.0.10.

  1. Try one workload. Give a low-stakes Deployment or CronJob its own DNS settings:
    spec:
      template:
        spec:
          dnsPolicy: None
          dnsConfig:
            nameservers: ["10.43.0.53"]
            searches: [media.svc.cluster.local, svc.cluster.local, cluster.local]
            options: [{name: ndots, value: "5"}]
    Use the workload's own namespace first in searches. When its pods restart, the workload appears on the Workloads tab, and its Service names still resolve (WrenDNS sends cluster.local on to CoreDNS).
  2. Make it the cluster's default. Point the kubelet at WrenDNS first and CoreDNS second, in /etc/rancher/k3s/config.yaml on every node, then restart k3s:
    kubelet-arg:
      - "cluster-dns=10.43.0.53,10.43.0.10"
    • Use kubelet-arg, not k3s's own cluster-dns option, which moves CoreDNS's Service too.
    • Restarting k3s leaves running pods running: each pod gets the new setting when it's next created. Move a namespace over when it suits you with kubectl -n media rollout restart deploy.
    • CoreDNS second keeps lookups working if WrenDNS's Service has no ready pods. Images based on Alpine (musl) ask both servers at once and take the first answer, so some of their lookups arrive through CoreDNS and are counted under CoreDNS.
    • For new clusters, have your install script write the file before it installs k3s:
      sudo install -D -m 0600 k3s-config.yaml /etc/rancher/k3s/config.yaml
      curl -sfL https://get.k3s.io | sh -
    • On other distributions, set clusterDNS in the kubelet's configuration file the same way.
    • WrenDNS's own pods keep asking CoreDNS (kubernetes.ownPodsUseClusterDns, on by default), so they never depend on their own Service.
  3. Give the nodes a second DNS server. CoreDNS sends everything outside the cluster to the node's own DNS servers, and so does the node itself. With only one, that server is a single point of failure for every pod. List two, such as the Pi and WrenDNS's LoadBalancer address. On Ubuntu with netplan:
          nameservers:
            addresses: [192.168.1.2, 192.168.1.53]
    Then sudo netplan generate && sudo nmcli connection reload && sudo nmcli device reapply <interface> applies it without taking the link down (NetworkManager), and kubectl -n kube-system rollout restart deploy/coredns makes CoreDNS read it.

Once the nodes use WrenDNS, leave kubernetes.podCidr and serviceCidr out. With them, WrenDNS sends reverse lookups for those ranges to CoreDNS, and CoreDNS passes the ones it doesn't know to the node's DNS servers, which now include WrenDNS: an unknown address's PTR would go round in a loop.

A Pi in the same cluster needs nothing. The chart's route to CoreDNS is marked node_only, so it stays on the pods in Kubernetes, and the Pi answers cluster.local as it would any other name. To undo the default, delete the kubelet-arg line and restart k3s.

Monitoring and encrypted DNS

  • Prometheus and Grafana: serviceMonitor.enabled scrapes the controller and every resolver pod, prometheusRule.enabled adds alert rules (down, all upstreams down, SERVFAIL rate, stale lists, masked client IPs, cluster health), and grafanaDashboard.enabled ships the dashboard for Grafana's sidecar.
  • Who may ask: networkPolicy.enabled limits who may query (dnsFrom), reach the UI (apiFrom), and scrape (metricsFrom).
  • Encrypted DNS for your devices: encrypted.dot.enabled (853), encrypted.doh.enabled (443, plus encrypted.doh.http3), and encrypted.doq.enabled add DoT, DoH, and DoQ to the DNS Service, with the certificate from encrypted.tls.secretName or a cert-manager issuer (encrypted.tls.certManager.issuerRef). It's reloaded when renewed.