Developers
Write UI tests
The web UI is tested against the real binary, in one browser session, in order. That is what makes the tests honest, and what makes them fragile if you are careless: a test that leaves something behind breaks a test that runs an hour of wall time later. This page is how to write one that doesn't.
How the suite is laid out
File in ui/tests/e2e/ | What it is |
|---|---|
1-ui.spec.ts | The main suite: one serial group on one server and one signed-in page. The first test creates the admin with the setup token; everything after runs as that admin. |
2-shots.spec.ts | Screenshots for the documentation. |
3-oidc.spec.ts | Sign-in through real Keycloak and Authentik (the oidc job). |
4-icons, 5-markers, 6-markup | Tests that need no browser page: pure functions and a scan of the source. Fast, and immune to everything on this page. |
server.mjs, dns.ts | The test server and a DNS helper. server.mjs writes a configuration, starts the binary with it on 127.0.0.1:18054 with the query log flushed every second, and runs a stub upstream that answers any name. query(name) asks the server over UDP and returns the response code. |
The configuration is ui/playwright.config.ts: one worker, no parallelism, and no retries.
The rules
- Prefer the test that needs no browser. Put the logic in a plain module and test it directly, as the icon tests do. A page test is for what only a page can show.
- Leave the server as you found it, or in a state no later test reads. Turn a setting back off (the two-factor test switches it on and then off again). Don't leave users, tokens, rules, or notes behind unless a later test needs them.
- Don't change what other tests count. The suite shares one query log and one cache. A name asked three times climbs the Cache page's top list (ranked by hits), and a later test that expected a different name first failed for exactly that reason. Ask a name once, or use one nobody else reads.
- Say what you depend on. The suite's earlier tests create things: the admin, the blocked name
ads.e2e.test, the local recordnas.e2e.test. If your test needs one, add a comment saying so; if you can, make the thing yourself. - Assert invariants, not counts of shared state. "A filter and its negation add up to every row in the window" holds whatever else is in the log; "there are 417 rows" does not.
- The test fails on any page error or console error, including a content-security-policy violation (only 401 and 403 are expected). Don't log errors on purpose.
- The title starts with the requirement ID (
obs_039 …), with a// REQ:comment above it, so the trace picks it up. - A failure hides everything after it. In a serial group the rest are "did not run", so the first failure is the real one. That is why there are no retries: a retry would run the group again on top of the state the first attempt left, and fail somewhere else.
Write and run a test
- Add it to
1-ui.spec.tsnear the feature's neighbours, or to a browser-less file if it needs no page. - Build the UI first, then the binary. The binary embeds
ui/dist, so a stale build tests old code:
The first-run test is in the pattern because yours needs the admin it creates.cd ui npm ci npm run build cargo build -p wrendns # from the repository root: cd .. first npx playwright install chromium # once npx playwright test tests/e2e/1-ui.spec.ts -g "first run|your test title"WRENDNS_BINpoints the server at another binary. - Run the whole file once before you push. A test that passes alone and fails in the suite (or the reverse) is the common failure of this kind. Run the whole
ui/tests/e2efolder too: the browser-less files are part of the job. - Break it on purpose. Put the bug back and watch the test fail. A test you have never seen fail proves nothing.
Time
- The query log is written about once a second in the test server. A row you just caused is not in the log yet: poll for it (
expect.poll), neversleep. - The API reads
fromandtoin whole seconds. A window that ends less than a second after your last row can exclude it. End a window in the past, a second and a half or more after the rows are in, and use one window for every count so a background query can't move them. - Background queries exist. The probes and checks write rows too. Count what you caused (by a name only you use), not the log.
- Sign-ins are slow. A password check is deliberately expensive. Expect seconds, and give the step a timeout of its own.
- A time-based credential (a two-factor code) is valid for a window of time, and the server accepts each step once. The two-factor test computes the code from the secret the page shows, then signs in with the next step's code, which the server accepts as clock skew and has not seen.
The checks
- The
uijob runs the suite against the real binary on every push; theoidcjob runs it again with real sign-in providers. - A flaky test is a bug. Re-run the failed job to get a green run, say that you did, and open an issue (#69 is an example with the likely cause for each). See How we work: the gates.
Worked examples
- Two-factor sign-in through the page. The two-factor box carried a pattern that no code could match (Svelte pitfalls), and no test touched it. The test turns two-factor on with a code computed from the page's secret (checking its own helper against the RFC's published test vector first), signs out, signs in with the next step's code, and turns it off. With the bug put back it fails at the first step, which is how you know it tests the bug.
- The query log's "not" filters. Three traps in one test. It asked a name three times (the Cache page's ranking test failed), it ended its window less than a second after its rows (the API's whole seconds), and it first relied on rows another test would have made. The final version asks each name once, waits three seconds and ends its window a second and a half in the past, and checks invariants: each filter plus its negation equals the whole window.