Application assurance auditor

Validate with the intended authorities

createAssuranceProgramValidationAuthorities composes the available authority from the framework basis, application capabilities and host facts. validateAssuranceProgramSpecification(program, authorities) returns consistency issues against that supplied context.

Calling the validator without authorities performs its structural checks; it does not provide the same authority-aware validation. Preserve the distinction in any application-creation or review integration.

Interpret capability applicability correctly

deriveExpectationCapabilityApplicability compares required capability references with available references. Its result is about configuration:

A source marker points to an implementation site. It neither enables a capability nor proves that a runtime path executed.

Inspect a real application’s consequences

This is a framework-repository inspection tool, not an installed application CLI command:

node scripts/assurance/report-capability-consequence.mjs \
  --app examples/wonder-crm --json

The report connects selected capabilities to the affected assurance expectations. Read it against the application’s current declarations; a historical result for another application or revision is not the current review scope.

Route clauses to their evidence sources

A requirement body carries clauses, evidence homes, execution planes, minimum evidence, verification steps and interpretation guidance. Homes such as DECLARED, EMITTED, OBSERVED, PERFORMED and APPLICATION_RECORDED identify where collection must go.

For example, an emitted security event belongs to its store of record. A performed review needs the review evidence. A queue execution record is not automatically evidence of an unrelated scheduled process.

Inspect the existing collection plan

From the framework repository:

node scripts/assurance/plan-evidence-collection.mjs \
  --app examples/wonder-todos \
  --requirement evidence-requirement:security-event-record \
  --json

This tool plans collection; it does not query the running application or judge the result. It resolves applicability first and groups the remaining clauses by evidence home.

The selected source below shows why an unavailable requirement is not treated like an empty successful collection. Comments are omitted.

if (applicability.status === 'NOT_APPLICABLE') {
  return {
    ...base,
    verdict: 'NOT_APPLICABLE_AS_CONFIGURED',
    clausesByHome: [],
    derived: true,
  };
}

const byHome = new Map();
for (const clause of demand.body?.clauses ?? []) {
  if (!byHome.has(clause.home)) byHome.set(clause.home, []);
  byHome.get(clause.home).push({
    clauseRef: clause.clauseRef,
    plane: clause.plane,
    obligation: clause.obligation,
    requiredAuditEventMembers: [...(clause.requiredAuditEventMembers ?? [])],
  });
}
return {
  ...base,
  verdict: null,
  derived: false,
  clausesByHome: [...byHome.entries()]
    .map(([home, clauses]) => ({ home, clauses }))
    .sort((left, right) => left.home.localeCompare(right.home)),
};

For requirements needing evidence, verdict: null preserves that collection and evaluation have not occurred. The plan is useful preparation without pretending to be a completed audit.

Collect from the authoritative source

Each evidence home has its own collector in the assurance cycle. Collectors that read the running application use a machine client holding only the audit-evidence reader role, with a short-lived token per step. Every measurement retains its source, target and time window, and none of them carries a verdict: the judge receives the evidence separately from collection.

A zero is recorded beside a positive control, so an empty window cannot pass for a quiet one. Acquiring state from external providers is still to come. An unavailable evidence export is a finding about evidence access until the underlying control’s behavior has been established.

Use the verdict vocabulary precisely

Only NONCONFORMITY opens a finding. OBSERVATION is not a deficiency, and INSUFFICIENT_EVIDENCE describes the evidence rather than the control, so neither opens one.

Learn from the full pilot sequence

The security-event pilot includes the original finding, calibration, an independent fresh-context judgment and recollection. The later review changed the interpretation; the final recollection concluded that the requirement was not applicable as configured.

That last conclusion rested on a capability gate that was itself corrected afterwards: the audit store the requirement reads exists in every application, so the requirement now applies and is judged in each cycle.

Read that sequence as evidence of the review method. Do not reproduce any single result as a statement about today’s deployment or omit the later results that superseded it.

Design recurring review around stable identity

A finding is keyed by requirement, deficiency class and clause scope. The key is derived from rule-based verdicts, so the same deficiency keeps the same identity from cycle to cycle; a model judge does not open findings. Reassessment updates that matter instead of flooding the operator with duplicates.

A finding takes its severity, MAJOR or MINOR, from the requirement; one whose rule states none is shown as ungraded, which is not the same as mild. Findings live in the application’s assurance repository, and each snapshot identifies the application and framework revisions. One notification per cycle summarizes what changed, marked urgent only when a major finding arrives.

Make review independence visible

Collection and judgment have different responsibilities. Fresh-context evaluation, independent authorship, model diversity where available and planted calibration cases address different sources of error.

A second, adversarial model judge can re-examine a requirement in a fresh context; it runs outside the scheduled cycle and records its independence on each axis. Preserve the achieved independence and calibration status in the result. An internal panel still belongs to the operator’s assurance process; external assessment remains a separate authority.