For agents: Skip to article. Documentation index. Read this page as Markdown.
Scull
Guide

Procedure: review and fix

Review extracted facts and attachments, resolve conflicts, and finalize schemas.

Read as Markdown

Every scull-schemafier run exits 2 when needs_review: true. CI should treat that as blocking.

What needs_review means

  • conflicts[]: spec and prose disagree on value_type, required, default, or enum_values. Spec wins in output; verify the other source.
  • caveat_evidence_changed: the quoted evidence text differs from the previous run (LLM variance or upstream edit). Usually safe to accept if the quote is still accurate.
  • extraction_failure: LLM call or validation failed for a prose file. Check the file and re-run.
  • unmapped_spec_field: spec field has no vocabulary entry. Add to vocabulary.yaml or ignore.
  • field_assignment_not_in_vocabulary: a committed or carried-over field is absent from the current vocabulary.
  • suspicious_field_assignment: a prose-backed field matches that vocabulary entry's explicit assignment_excludes; inspect it and correct either the field assignment or the reviewed constraint.

Warnings and conflicts are never accepted implicitly. A reviewer may correct the artifact, or retain an expected diagnostic and record an explicit approval with finalize --approve as described below.

Audit

scull-schemafier audit-assignments \
  --schemas schemas \
  --vocab vocabulary.yaml

This deterministic audit checks every selected committed schema, makes no LLM call, does not modify files, and exits 2 if it finds a suspicious or out-of-vocabulary assignment. Use repeated --concept options to limit its scope.

Staleness check (CI gate)

scull-schemafier check \
  --concept myconcept \
  --spec data/myconcept/spec.json \
  --prose data/myconcept/prose \
  --out schemas/

Exits 1 if any source file changed since the schema was last generated. Gate on this before merging.

Complete manual review and release

For a concrete example, suppose the authoritative source says a timeout field is measured in seconds, while the extracted description says milliseconds. Correct the description and any affected caveat text, verify the quoted evidence against the source, and check that the caveat's applies_to contains the actual field name timeout. Record the source and the unit correction in the approval note. The commands below finalize that edited artifact without running extraction again and potentially replacing the correction.

  1. Open the generated schema and its cited source locations. Verify field meanings, structural values, caveat summaries and evidence, attachments, and whether anything material is missing. scull validate cannot perform this semantic review.

  2. Resolve each warning and conflict. Correct a field, caveat, attachment, or source when the artifact is wrong. A diagnostic may remain only when the reviewer has confirmed the output is correct despite it and is willing to record that decision.

  3. After any manual edit, refresh the content hash without extraction or an LLM:

    scull-schemafier finalize --schemas schemas --concept myconcept
scull validate

    Repeat --concept to finalize several concepts, or omit it to process the entire schemaset. Rehashing does not approve an artifact and does not clear needs_review.

  4. Record approval only after the review is complete:

    scull-schemafier finalize \
  --schemas schemas \
  --concept myconcept \
  --approve \
  --reviewer "reviewer@example.com" \
  --note "Verified fields, evidence, attachments, and retained diagnostics"

    --approve requires non-empty --reviewer and --note. It explicitly accepts the current warnings and conflicts, sets needs_review: false, refreshes schema_hash, and records a timestamped review fingerprint. It records a human decision; it does not prove factual accuracy.

  5. Gate and build the release artifacts together:

    scull validate --release
scull-schemafier kg build --release --schemas schemas --out kg

    The release checks require a non-empty schemaset, current hashes, needs_review: false, and a matching recorded approval for every schema. Ordinary kg build remains available for local development with unapproved artifacts.

Deploy and roll back as a pair

Deploy the approved schemas/ directory together with kg/CURRENT and the complete generation it names. Search refuses to combine schema content with a manifest built from different content, before it calls the query embedding provider. Recreate or reload long-lived providers after switching the deployed pair; they intentionally pin loaded schemas and an index generation.

For rollback, restore both the prior schemaset and its matching index generation, then reload the consumer. Do not point a new schemaset at an old index or restore only CURRENT. Retain the previous pair until the new deployment has passed its lookup and search smoke checks.