See whether sources reach the result
A well-written answer can still leave its supporting sources behind. Wildo adds a simple observation beside model review: how many collected source addresses also appear in a produced artifact.
That makes missing citations visible without asking another model to estimate them. The observation points to work worth inspecting; it does not establish that a cited page supports a claim.
Example — Notice evidence missing from a market brief
Research identifies relevant pages, but the resulting market brief contains no citations. The observation exposes that difference even when the prose reads confidently and the reviewer likes it.
For engineers
The choreography result carries a provenance entry for a child that lands content when source URLs are present in step results accumulated before that round starts. evidenceUrlCount is the distinct collected-address count; citedUrlCount is the intersection with addresses found in the child’s landed values.
Before dispatching the round, the runtime snapshots source addresses with const priorEvidenceUrls = collectProvenanceUrls([...stepResults.values()]). Every child in that round is compared with this same set, so its own returned sources and concurrent sibling results cannot count retroactively.
This selected excerpt shows the subsequent comparison:
if (childLanded.length > 0) {
const evidenceUrls = priorEvidenceUrls;
if (evidenceUrls.size > 0) {
const citedUrls = collectProvenanceUrls(settlement.value.landedValues ?? []);
provenance.push({
playbookRef: child.playbookRef,
evidenceUrlCount: evidenceUrls.size,
citedUrlCount: [...citedUrls].filter((url) => evidenceUrls.has(url)).length,
});
}
}
collectProvenanceUrls scans serialized values rather than relying on one family’s citation property. An address anywhere in the value can count. This accommodates different artifact shapes, while making the signal broader than clause-level citation validation.
Read the numbers as a prompt to investigate
| Observation | Useful next question |
|---|---|
| Sources present, none cited | Did the output omit useful support? |
| Some collected sources appear | Which claims use them, and do the pages support those claims? |
| No observation | Was there source-bearing step output available for this comparison? |
The observation does not reject a run. Some outputs legitimately have no place for citations; a creator’s own requirements may also be sufficient input. The result should lead the reader back to the source material and the actual artifact.
This is an address-presence check over prior-round returned values. It does not certify when a source was discovered, whether the source was read, or whether its contents are reliable. Use claim-level evidence review for those questions, alongside the advisory reviewer.