Engine behaviour end-to-end harness

@wildo-ai/engine-behavior-e2e is the suite that answers “does the engine actually work”. Fifty named scenarios are registered in one file; each is a script that provisions its own organisations against a running application, drives one engine capability over real HTTP or a real browser, reads the result back from the other side of a process boundary, and tears down. A run is green only when every assertion passed and the backend logged nothing fatal while they ran.

The problem it solves

Most of what this framework claims is about behaviour under conditions a unit test cannot create: two tenants at once, a signed assertion arriving from outside, a job that has to survive a broker, a token minted for one surface being refused on another. A test that mocks the boundary proves that the mock behaves; it says nothing about the running system.

The second problem is subtler and cost more. A suite can pass every assertion it makes while the application under it is failing at something the suite never looked at, so the report is green and the product is broken. The harness treats that as the default hazard rather than an edge case: the run window is watched independently of the assertions, and a run in which the backend screamed is not a passing run no matter what the scenarios concluded.

What it does

Fifty registered scenarios drive a running stack the way an outside system would, and one command is the bar.

A scenario is a row carrying a stable identifier, the script that replays it, the lane it asserts against, and the runtime surfaces it needs ready. Forty-seven sit on the application-runtime lane and three on the control-plane lane. A third lane is declared and deliberately unused, so the tag is honest about being one of several on one substrate rather than pretending the substrate is single-purpose.

Everything application-specific sits behind ONE seam — an adapter shipped inside the application, injected as a subprocess whose working directory and command resolve from environment bindings rather than from a hardcoded path. A scenario therefore names capabilities, never the application, and a second application slots in by shipping its own adapter.

The gated command attaches to the stack a developer is already running, gates on health, pins the current end of the development log, runs the selected scenarios, then scans only what was appended during the window. The gate matches the fatal token on the RAW line, in both log shapes, because a parser that reads only structured records would certify a human-readable run as clean. It also matches criticals a caller caught by contract and logged at warning level — which is where a fire-and-forget dispatcher hides its own breakage.

Because every scenario opens with a destructive pre-clean keyed on a deterministic reference, two runs against one stack would delete each other’s fixtures mid-drive. A second run is refused by name, and the claim is held by the process TREE so a runner’s own subprocesses inherit it. The marker is honoured only while the owning process is alive and under two hours old, so a hard-killed run cannot wedge the lane.

What you decide

Adding a scenario is a script and one registry row. The harness supplies the adapter, the attach, the surface, provisioning, authentication and teardown — so an ordinary API scenario is just the pipeline of steps.

Limits

The gated command with no arguments does not run everything registered. The plan intersects the registry with what the application’s own adapter declares. The difference is a subset by DECISION rather than by drift: each of the six left out carries a written reason — an equivalent lane already gates the same contract, a real external fleet or model budget is spent, or the surface is one most changes do not touch.

Two scripts sit beside the registry with no row, and both are deliberate: one is a superseded proof kept as a runnable historical record, the other a stack-independent self-test of the harness’s own plumbing. Neither runner can reach an unregistered script, so both run only if someone invokes the file directly — which is what they are for.

One application implements the adapter. The seam is built for more and is exercised by one, so “works across applications” is a property of the design rather than a measured result.

The suite attaches to a stack; it does not diagnose one.