# Procedure: review and fix

> Documentation index: [llms.min.txt](https://caseyrfsmith.github.io/scull/llms.min.txt) · Document manifest: [documents.json](https://caseyrfsmith.github.io/scull/documents.json)

Breadcrumbs: Build and use > Procedure: review and fix

Every `scull-schemafier run` exits 2 when `needs_review: true`. CI should treat that as blocking.

## What `needs_review` means

- `conflicts[]`: spec and prose disagree on `value_type`, `required`, `default`, or `enum_values`. Spec wins in output; verify the other source.
- `caveat_evidence_changed`: the quoted evidence text differs from the previous run (LLM variance or upstream edit). Usually safe to accept if the quote is still accurate.
- `extraction_failure`: LLM call or validation failed for a prose file. Check the file and re-run.
- `unmapped_spec_field`: spec field has no vocabulary entry. Add to `vocabulary.yaml` or ignore.
- `field_assignment_not_in_vocabulary`: a committed or carried-over field is absent from the current vocabulary.
- `suspicious_field_assignment`: a prose-backed field matches that vocabulary entry's explicit
  `assignment_excludes`; inspect it and correct either the field assignment or the reviewed constraint.

Warnings and conflicts are never accepted implicitly. A reviewer may correct the artifact, or retain
an expected diagnostic and record an explicit approval with `finalize --approve` as described below.

## Audit

```bash
scull-schemafier audit-assignments \
  --schemas schemas \
  --vocab vocabulary.yaml
```

This deterministic audit checks every selected committed schema, makes no LLM call, does not modify
files, and exits 2 if it finds a suspicious or out-of-vocabulary assignment. Use repeated
`--concept` options to limit its scope.

## Staleness check (CI gate)

```bash
scull-schemafier check \
  --concept myconcept \
  --spec data/myconcept/spec.json \
  --prose data/myconcept/prose \
  --out schemas/
```

Exits 1 if any source file changed since the schema was last generated. Gate on this before merging.

## Complete manual review and release

For a concrete example, suppose the authoritative source says a `timeout` field is measured in
seconds, while the extracted description says milliseconds. Correct the description and any
affected caveat text, verify the quoted evidence against the source, and check that the caveat's
`applies_to` contains the actual field name `timeout`. Record the source and the unit correction in
the approval note. The commands below finalize that edited artifact without running extraction
again and potentially replacing the correction.

1. Open the generated schema and its cited source locations. Verify field meanings, structural
   values, caveat summaries and evidence, attachments, and whether anything material is missing.
   `scull validate` cannot perform this semantic review.
2. Resolve each warning and conflict. Correct a field, caveat, attachment, or source when the
   artifact is wrong. A diagnostic may remain only when the reviewer has confirmed the output is
   correct despite it and is willing to record that decision.
3. After any manual edit, refresh the content hash without extraction or an LLM:

   ```bash
   scull-schemafier finalize --schemas schemas --concept myconcept
   scull validate
   ```

   Repeat `--concept` to finalize several concepts, or omit it to process the entire schemaset.
   Rehashing does not approve an artifact and does not clear `needs_review`.
4. Record approval only after the review is complete:

   ```bash
   scull-schemafier finalize \
     --schemas schemas \
     --concept myconcept \
     --approve \
     --reviewer "reviewer@example.com" \
     --note "Verified fields, evidence, attachments, and retained diagnostics"
   ```

   `--approve` requires non-empty `--reviewer` and `--note`. It explicitly accepts the current
   warnings and conflicts, sets `needs_review: false`, refreshes `schema_hash`, and records a
   timestamped review fingerprint. It records a human decision; it does not prove factual accuracy.
5. Gate and build the release artifacts together:

   ```bash
   scull validate --release
   scull-schemafier kg build --release --schemas schemas --out kg
   ```

   The release checks require a non-empty schemaset, current hashes, `needs_review: false`, and a
   matching recorded approval for every schema. Ordinary `kg build` remains available for local
   development with unapproved artifacts.

## Deploy and roll back as a pair

Deploy the approved `schemas/` directory together with `kg/CURRENT` and the complete generation it
names. Search refuses to combine schema content with a manifest built from different content, before
it calls the query embedding provider. Recreate or reload long-lived providers after switching the
deployed pair; they intentionally pin loaded schemas and an index generation.

For rollback, restore both the prior schemaset and its matching index generation, then reload the
consumer. Do not point a new schemaset at an old index or restore only `CURRENT`. Retain the previous
pair until the new deployment has passed its lookup and search smoke checks.