Review the quality, keep the reasoning
A valid structure does not tell you whether a proposal is convincing or a requirement reflects the product’s goals. Wildo can ask a reviewer to examine produced work against criteria defined by its playbook.
Each finding includes a reason the team can inspect and challenge. The review informs the next decision; it does not replace deterministic checks or automatically approve the work.
Example — Check whether a roadmap serves the brief
A roadmap may be well formed yet spend its first release on secondary concerns. The reviewer compares its priorities with the creator’s stated goals and explains where they diverge.
For engineers
A playbook’s evaluation treatment owns its criteria, instructions and verdict guidance. Describe observable differences between a pass, concern, indeterminate result and gap. The runtime builds a structured output contract from those declared criterion references.
The brief combines landed family values with creator input when available. Computed context, such as the coherence report, is supplied separately and identified as deterministic evidence. The reviewer should interpret what those facts mean, not invent a replacement calculation.
Keep every criterion visible
This runtime excerpt shows how an omitted answer remains visible and how the overall verdict is derived:
// One verdict per declared criterion; a criterion the model skipped is INDETERMINATE, honestly.
const byRef = new Map<string, PlaybookJudgeCriterionVerdict>();
for (const entry of parsed.criteria) {
const previous = byRef.get(entry.criterionRef);
// A repeated criterion is not permission to erase an adverse assessment. Preserve every
// rationale and the most severe verdict, independent of the provider's answer order.
byRef.set(entry.criterionRef, previous ? {
criterionRef: entry.criterionRef,
verdict: VERDICT_SEVERITY[entry.verdict] > VERDICT_SEVERITY[previous.verdict] ? entry.verdict : previous.verdict,
rationale: `${previous.rationale}\n\nAdditional verdict (${entry.verdict}): ${entry.rationale}`,
} : entry);
}
const criteria: PlaybookJudgeCriterionVerdict[] = criterionRefs.map((criterionRef) => {
const entry = byRef.get(criterionRef);
return entry
? { criterionRef, verdict: entry.verdict, rationale: entry.rationale }
: { criterionRef, verdict: MetaWorkflow_EvaluationVerdict.INDETERMINATE, rationale: "The judge returned no verdict for this criterion." };
});
const overall = criteria.reduce<MetaWorkflow_EvaluationVerdict>(
(worst, entry) => (VERDICT_SEVERITY[entry.verdict] > VERDICT_SEVERITY[worst] ? entry.verdict : worst),
MetaWorkflow_EvaluationVerdict.PASS,
);
const result: PlaybookJudgeResult = { judged: true, definitionRef: input.treatment.definitionRef, criteria, overall };
Repeated entries for one criterion retain the most severe verdict and all their explanations, regardless of order. An unknown criterion is refused by the output schema.
INDETERMINATE means the review did not establish an answer. It must not disappear from the report or become a pass by omission. The overall result follows the runtime severity ordering, while the per-criterion rationale remains the useful material for a correction.
Read the result in the right place
| Result | Meaning for the caller |
|---|---|
judged: true | Inspect criteria, their rationales and overall |
judged: false | Inspect skipped; there is no review verdict |
| Landed output | Already exists independently of the advisory verdict |
The generation host invokes the judge after landing, and the companion also exposes a dedicated judge route. Failures inside the model-review path return a skipped result. A missing creator brief leaves less evidence for questions about intent; neither a confident rationale nor a passing verdict establishes facts the reviewer never received.
Use family validation for checks that can be computed and source observations to inspect citation presence.