Mechanism:
-
All state data is persisted in a versioned, append-only event store to ensure immutable history and traceability.
-
Primary store: JSON document database, logically partitioned by Job ID and Step ID for efficient sharding and retrieval.
-
Versioning is achieved through a hybrid approach using timestamp + delta hash indexing; this ensures both temporal traceability and structural consistency. Previous states remain retrievable via version tags, enabling full audit trail regeneration.
-
Cognition memory graphs are version-snapshotted per transition checkpoint and stored in parallel to document state evolution and support explainable transitions.
State Objects:
-
Step outputs (raw and processed), including inference traces and normalized text segments.
-
Confidence vectors reflecting system-assessed reliability scores.
-
Provenance chains detailing data lineage and source transformation paths.
-
Contradiction flags used to highlight semantic or logical inconsistencies.
-
Adjudication annotations (from Vault) capturing human oversight or exception decisions.
-
Cognition layer snapshots (Keymaker), preserving decision-making structures and interim reasoning graphs.
-
Additional indexing of metadata for query optimization during large-scale audits, including audit trail checkpoints, mutation lineage tags, and retrieval acceleration hints for forensic queries.