# Schema file format > Documentation index: [llms.min.txt](https://caseyrfsmith.github.io/scull/llms.min.txt) · Document manifest: [documents.json](https://caseyrfsmith.github.io/scull/documents.json) Breadcrumbs: Reference > Schema file format A `.schema.yaml` file is the output of one pipeline run on one concept. It's the contract between the scull-schemafier and the CLI. The format is stable and human-reviewable — you're meant to read and diff these files. ## Top-level structure ```yaml schema_version: "0.1" concept: caveat schema_hash: 3306230a76d71d81... # SHA-256 of all content fields _meta: generated_at: "2026-09-04T15:51:40Z" review: reviewer: reviewer@example.com note: Verified fields, evidence, attachments, and retained diagnostics approved_at: "2026-09-09T18:30:00+00:00" fingerprint: 5b9a6f... sources: spec: url: data/caveat/spec.json hash: a7b8c74d... status: ok prose: - url: https://docs.example.com/caveat-format hash: 78235a9a... status: ok fields: id: value_type: string required: true default: null enum_values: null description: "Stable kebab-case identifier derived from..." confidence: high sources: [spec, prose] locator: url: https://docs.example.com/caveat-format heading: Caveat identifiers caveats: - id: id-never-changes-automatically summary: "Caveat IDs are deterministic but never automatically updated..." severity: medium sources: [prose] confidence: high evidence: "The ID is derived once at creation time and stored..." evidence_hash: 17baf96c0493 applies_to: [id] applies_when: [operation.update] locator: url: https://docs.example.com/caveat-format heading: Caveat identifiers structure: # Full type hierarchy from OpenAPI specs — walks endpoints backwards bill.BillOutData: # {tag}.{SchemaName} — tag comes from operation tags type: object required: [billNumber, netAmount, billDate] properties: billNumber: type: string minLength: 1 maxLength: 50 netAmount: type: number minimum: 0 needs_review: false conflicts: [] warnings: [] deprecated: false ``` *** ## Field reference ### `schema_version` Format version. Readers accept `"0.1"` and `"0.2"`; the CLI validates this on load. JSON Schema constraint artifacts require `"0.2"`. Older readers that only support `"0.1"` reject them rather than silently ignoring their constraints. ### `constraint_bundle` (format `0.2`) A reference to a digest-bound closure manifest under `constraints/`, with each original JSON resource stored once by content digest. Distribute that directory alongside the `.schema.yaml` files. Readers verify paths and hashes locally; no source checkout or network is required. Missing or altered resources are errors. Interpretation records distinguish supported rules, annotations, and rules preserved without interpretation. Partial coverage stays partial after review approval. Use `scull inspect-schema [json-pointer] --limit 100` to inspect original rules, interpretation status, and reference links. `get` and `context --detail` expose complete scoped records; default `context` reduces them to concise constraints, requiredness, completeness, and inspection links. Ambiguous property aliases retain their distinct rule identities and candidates. Property fields may also expose `default`, `example_values`, and `runtime_transforms`. Defaults and examples are JSON Schema annotations. `runtime_transforms` records application-specific behavior declared by the source (for example trimming a string); it is surfaced for authoring but remains a nonstandard keyword, so the constraint bundle does not claim that ordinary JSON Schema validation reproduces it. ### `concept` Canonical concept name. Must match a top-level key in `vocabulary.yaml`. This is what you pass to `scull get`, `scull caveats`, etc. ### `schema_hash` SHA-256 of stable content fields, excluding `schema_hash`, `_meta`, `warnings`, and `conflicts`. Warnings may reflect non-deterministic extraction diagnostics, while conflicts are derived from already-hashed fields. `scull validate` recomputes and compares this hash. The hash detects accidental content mismatch; it is not a signature and does not prove authenticity or correctness. This is the primary change signal in the system: - KG builder reuses a concept's rows when its provider identity, chunker version, and exact embedded-text fingerprint are unchanged - `scull-schemafier check` (CI staleness) compares source hashes, not `schema_hash` — those are two different signals Deliberately not based on `generated_at`. Regenerating with no content changes produces the same hash — nothing downstream re-processes. ### `_meta` Non-content metadata. `generated_at` is an ISO 8601 timestamp. An approved artifact also has a `review` mapping containing non-empty `reviewer`, `note`, `approved_at`, and `fingerprint` strings. `finalize --approve` creates that record. The review fingerprint binds the schema's content together with its current warnings and conflicts, excluding only `_meta` and `schema_hash`. Any later edit to content, diagnostics, or review state invalidates release approval until a reviewer approves the new state. The fingerprint records what was reviewed; it is not a cryptographic identity or a semantic validation result. ### `sources` Hashed references to every input consumed during the pipeline run. **`sources.spec`:** - `url` — path or URL of the spec JSON file - `hash` — SHA-256 of the file bytes at extraction time - `status` — `ok`, `missing`, or `stale` **`sources.prose[]`:** - `url` — canonical URL (from `--source-url` flag, or local file path as fallback) - `hash` — SHA-256 of the file bytes at extraction time - `status` — `ok`, `missing`, or `stale` - `signals` — metadata mapped from YAML frontmatter via `scull.yaml metadata_mappings`. Common signals: `source_url`, `last_verified`, `owner`. `status: missing` means the source URL was unreachable on the most recent re-fetch. The schema and its content are preserved — a missing page may be temporary. `deprecated: true` is a separate, human-set flag. ### `fields` Map of canonical field name → field entry. Keyed by the canonical name from `vocabulary.yaml`, not whatever the source happens to call it. **Field entry properties:** | Property | Source wins | Notes | | ------------- | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `value_type` | Spec | e.g. `boolean`, `string`, `map[string]string`, `integer`, `array` | | `required` | Spec | `true` or `false`. Nullable types (`["string", "null"]`) are collapsed to the non-null type before storage. | | `default` | Spec | Any value or `null` | | `enum_values` | Spec | List of allowed values, resolved through $ref chains. If a property's type is `$ref`'d to an enum schema, the enum values follow. | | `constraints` | Spec | Array of constraint strings (e.g., `"minimum=0"`, `"maxLength=50"`, `"pattern=^[A-Z]{3}$"`). Present for fields extracted from OAS specs; omitted for prose-only fields. | | `description` | Prose | Behavioral meaning, not just type definition | | `confidence` | Derived | `high` = both sources present; `medium` = one source | | `sources` | Derived | `[spec]`, `[prose]`, or `[spec, prose]` | | `locator` | Prose | `{url, heading}` pointing to the doc section; omitted for spec-only fields | **Reconciler rules in plain English:** - Spec wins on everything structural: `value_type`, `required`, `default`, `enum_values` - Prose wins on everything behavioral: `description` and all caveats - Description differences between spec and prose are not conflicts — prose always wins silently - Structural disagreements between spec and prose → `conflicts[]` entry + `needs_review: true` ### `caveats` List of behavioral caveat entries. Always extracted from prose — the spec never emits caveats. Caveats are the core value of Scull for agent use. A caveat is anything that would surprise a developer: a non-obvious constraint, a silent failure mode, an exception to the general rule, behavior that contradicts intuition. **Caveat entry properties:** | Property | Notes | | --------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | | `id` | Stable kebab-case identifier. Derived once, never automatically changed. | | `summary` | One sentence, written for an engineer scanning a list. | | `severity` | See severity scale below. | | `sources` | Always `[prose]`. | | `confidence` | `high` (clear, specific) or `medium` (inferred or ambiguous). | | `evidence` | Verbatim quote from the source, max 500 chars. Proves the caveat is real. | | `evidence_hash` | SHA-256 of normalized evidence, first 12 hex chars. Used for drift detection. | | `applies_to` | List of field IDs this caveat relates to. `null` if it applies to the concept generally. | | `applies_when` | Optional, review-owned list of lowercase dot-delimited situation tags, such as `consume.env` or `consume.volume.subpath`. Omit for an unconditional caveat. | | `locator` | `{url, heading}` pointing to the exact doc section. | `applies_when` refines field-level applicability without semantic inference. Tags form a hierarchy: a request for `consume.volume.subpath` also matches the broader `consume.volume` tag. Multiple tags on one caveat are alternative situations. The scull-schemafier does not invent these tags; a reviewer adds them to a committed schema, and reconciliation preserves them when the caveat keeps its stable ID. Calling `scull context` without situation tags remains conservative and returns the complete set. **Severity scale:** | Severity | Use when | | -------- | ------------------------------------------------------------------------------------------------------------- | | `high` | Silent failure, data loss, security issue, or the behavior directly contradicts what a developer would assume | | `medium` | Causes confusion, extra debugging time, or requires knowing the exception to the rule | | `low` | Minor surprise, cosmetic difference, or common-but-fine misunderstanding | **ID stability:** caveat IDs are derived from the first 5 significant words of the summary plus a 6-hex SHA-256 suffix. They're stable across re-runs even if the LLM rephrases slightly. To change an ID, edit it directly in the schema file — it will not be automatically regenerated. **Evidence drift:** on re-runs, if the LLM quotes slightly different text than last time, `evidence_hash` changes and a `caveat_evidence_changed` warning is emitted with `needs_review: true`. This is expected behavior for re-runs where the source docs didn't change — accept it after verifying the evidence is still accurate. When an unchanged prose file is skipped, its existing fields and caveats are carried forward without an execution-state marker. If a current spec/OAS run re-emits one of those fields, its prior prose description and locator are restored while the current structural values remain authoritative. This makes the first cached rerun and later cached reruns preserve both content and `schema_hash`. ### `needs_review` `true` when the reconciler found at least one: - Conflict between spec and prose on `value_type`, `required`, `default`, or `enum_values` - `caveat_evidence_changed` warning (evidence hash drifted from stored value) - `extraction_failure` or `empty_candidates` warning that might have omitted content Unresolved diagnostics tied to a skipped source (or not safely attributable to a source) remain visible during cache hits. A failure tied to a successfully reprocessed source can clear, even if another source was skipped in the same run. Informational notices such as `thin_source_skipped` do not set this flag by themselves. The schema is still written and usable for development. The scull-schemafier exits with code 2. CI pipelines should treat exit code 2 as blocking. A reviewer can correct the artifact or explicitly accept its current warnings/conflicts with `finalize --approve`; release validation then verifies that the recorded decision still matches the artifact. ### `conflicts` List of cross-source disagreements on structural properties. Spec wins in the output, but the conflict is recorded for human review: ```yaml conflicts: - property: value_type field_id: threshold spec_value: integer prose_value: float ``` Only `value_type`, `required`, `default`, and `enum_values` produce conflicts. Description differences do not. ### `warnings` Extractor and reconciler warnings. Nothing is silently dropped — if something went wrong during extraction, it appears here. | Kind | Meaning | | ------------------------- | ------------------------------------------------------------------------------- | | `extraction_failure` | LLM call or schema validation failed for a prose file | | `empty_candidates` | Every model in the fallback chain returned zero usable candidates | | `thin_source_skipped` | Near-empty prose file was deliberately skipped to avoid hallucinated extraction | | `no_prose_files` | Prose directory contained no markdown files | | `unmapped_spec_field` | Spec field has no matching entry in vocabulary | | `caveat_evidence_changed` | Evidence hash differs from stored value (drift detection) | | `source_missing` | Source URL was unreachable on re-fetch | ### `deprecated` `true` only when upstream spec or prose explicitly marks the concept deprecated. This is a deliberate human action — not automatically set when a source goes missing or a URL returns 404. *** ## Structure block (OAS ingestion) When using OAS-based schema generation (`scull-schemafier run --oas `), a `structure:` block captures the full extracted type hierarchy from the OpenAPI spec. It's a **flat** dict keyed by (optionally normalized — see below) schema name, not nested by endpoint tag: ```yaml structure: CreateOrderRequest: # Schema name resolved via $ref (post-normalization) type: object required: [amount, currency] properties: amount: integer ftype: string endpoint: POST /orders examples: CreateOrderRequest.amount: [50000] ``` Every structure entry — whether it came from a `$ref`-derived body, an inline (non-`$ref`) body, or a response body — is written in this same flat shape (`type`/`properties`/`required` directly alongside `endpoint`/`examples`, plus `response_code` for response-derived entries). `scull shape` also transparently normalizes the older `{"flattened": {...}}`-wrapped shape at display time, for any schema file written by a version of the parser before this was unified — but the parser itself never produces that shape anymore. Field naming (in the `fields:` block, not `structure:`) is deterministic — `{tag}.{SchemaName}.{prop}` comes straight from the OAS operation's tags and $ref resolution. No manual mapping needed; group resolution happens through tree-walking endpoints where tags and operation metadata provide context. The parser resolves all $ref chains with cycle detection (including two fields that reference the *same* shared schema — that's not a cycle), flattens allOf composition, collapses `oneOf[string, null]` into `string`, and follows enum schemas through refs. The `structure:` block enables standalone JSON payload lookups without needing an external OpenAPI service. Agents can query this directly via `scull shape [schema_name]` (zero inference, same as `get`/`caveats`) to get exact request/response shapes, while the `fields`/`caveats` layer answers behavioral questions about when to use X vs Y and what breaks silently. #### Schema name normalization For APIs with noisy or verbose schema naming (e.g., `QueryPayoutRecordsItem`, `TransferOutDetailVendor`), you can configure regex normalization rules in `scull.yaml`: ```yaml oas_normalize: rules: - match: "(?i)(Query|Result|Search).*Record" rename: "\\1" - match: "(.*)DetailRecord$" rename: "\\1.Detail" ``` Rules apply in order; each schema name matches at most the first rule that fires and the result is used for both `fields:` field\_ids and `structure:` keys. There is no separate persistence step or `__oas_mappings__` block—rules are applied fresh from `scull.yaml` on every run, so a rule change takes effect immediately on the next `scull-schemafier run --oas`. No code path reads or writes `__oas_mappings__` today; if that key appears in an older schema file, it is inert and safe to delete. When no rules are configured, raw schema names pass through unchanged—no assumptions, no forced grouping.