Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

The JSON report, field by field

The goal of this page: a consumer can trust the JSON without reading the code. The formal schema lives at schemas/report-v0.4.schema.json and the integration suite validates every emitted document against it; this page explains what the fields mean.

Versioning

The document carries schema_version, versioned independently of the tool: additive changes bump the minor (and ship a new schema file), breaking changes bump the major. Consumers should:

  • branch on stable field names and assertions[].name, not on positions;
  • tolerate unknown fields (a newer minor may add some);
  • check schema_version if they depend on a specific shape.

Always present

FieldTypeMeaning
schema_versionstringSchema of this document (0.4.x)
tool_versionstringdelta-explain release
elapsed_msintegerWall-clock analysis duration
tablestringTable path/URI as given
versionintegerDelta snapshot version analyzed
predicatestring | nullThe --where input, null without one
total_filesintegerFiles before pruning
final_filesintegerFiles surviving the last phase
total_pruning_pctnumberOverall reduction in percent
analysisobject | nullPredicate classification (partition_safe, partition_exact, stats_safe, unsplittable, confidence, notes), null without --where
table_featuresobjectDetect-and-declare block: deletion vectors, column_mapping_mode, clustering_columns, in_commit_timestamps, unrecognized_writer_features, and warning notes
statsobjectStatistics coverage: mode is exact / partial / absent
phasesarrayChained pruning phases, empty without --where
assertionsarrayOne entry per gate that ran, empty without gate flags
result"pass" | "fail" | nullOverall gate outcome, null when no gates ran

Only with --verbose

FieldTypeMeaning
filesarrayPer-file outcomes, capped by --limit
files_truncatedbooleanTrue when --limit cut the array short

Only with --explain-why

FieldTypeMeaning
explainarrayDiagnoses of why the predicate pruned as it did: {code, severity, message, suggestion}. Stable codes: NO_PARTITION_FILTER, WEAK_DATA_SKIPPING, STATS_ABSENT, UNSUPPORTED_FRAGMENT

Per file: path, size_bytes, partition_values (string map), num_records (nullable; includes soft-deleted rows under deletion vectors), has_stats, kept, pruned_by.

  • kept: true means the file survives every phase. The set of kept files is the survivor set: guaranteed to be a superset of the files any engine needs for this predicate (see semantics).
  • pruned_by names the first phase that eliminated the file (one of the phases[].name values), or null when kept. Phases chain, so "first phase whose survivor set misses the file" is exactly the phase that eliminated it.

confidence

Appears globally (analysis.confidence) and per phase. It labels how precisely the elimination can be explained, never whether it is correct:

  • exact — provably precise elimination (partition values compared directly);
  • conservative — sound but possibly loose (min/max ranges can overlap a bound without the file containing a match);
  • incomplete — part of the predicate could not be attributed (mixed OR, unsupported construct); totals remain sound, a note says why.

Stable note codes

Notes are {code, message} pairs. The message text may be reworded at any time; the code values are stable identifiers, and new codes are additive:

CodeWhereMeaning
UNSPLITTABLE_ORanalysis.notesAn OR mixes partition and non-partition columns; routed conservatively
UNSUPPORTED_EXPRESSIONanalysis.notesA fragment is outside the pruning language; it keeps all files, siblings still prune
PARTITION_EVAL_GAPanalysis.notesSome partition values could not be evaluated against the partition_exact fragment; the affected files are kept
DELETION_VECTORStable_features.notesFiles carry deletion vectors; record counts include soft-deleted rows
COLUMN_MAPPINGtable_features.notesThe log stores physical column names; verbose stats may display them
LIQUID_CLUSTERINGtable_features.notesThe table is liquid-clustered; layout is not directory partitions, data skipping still applies
UNRECOGNIZED_TABLE_FEATUREtable_features.notesThe table carries writer features this tool does not know; the numbers do not account for whatever they imply

assertions[]

Discriminated by name:

  • {"name": "min_pruning", "threshold": N, "actual": N, "result": "pass"|"fail"}
  • {"name": "stats_complete", "missing_count": N, "result": "pass"|"fail"}

result at the top level aggregates them; a fail also makes the process exit 1.

Error contract

On any runtime error stdout is empty and the message goes to stderr, so delta-explain ... --format json | jq ... never parses a partial document. The full exit-code table is in semantics.

Consumer patterns

# gate on the overall result
delta-explain ./t -w "..." --min-pruning 80 --format json | jq -e '.result == "pass"'

# survivor set as a file list
delta-explain ./t -w "..." --format json --verbose | jq -r '.files[] | select(.kept) | .path'

# fail a pipeline when deletion vectors appear
delta-explain ./t --format json | jq -e '.table_features.deletion_vectors.files_with_deletion_vectors == 0'