Evidence Verification Kit
Check the evidence yourself
A DeepSweep evidence ledger is a hash-chained record of the decisions made on an AI agent's tool calls: allowed, denied, or held for approval. Each entry can also carry its own signature, when the gateway that writes it is given a signing key. This kit lets you check a ledger without trusting us: in this page, or offline with two short Python files and nothing to install. It also says, as plainly as what a pass proves, what a pass does not prove.
What is available today
The kit is. The format, both checkers, the golden vectors and a signed sample ledger are published here, and the checkers run on any machine with Python 3.8 or later.
The component that writes this ledger in real use is not yet generally available. The sample below is synthetic: an invented agent session written in the same format, signed with a key created for it and then discarded, so nobody, us included, can add an entry that checks out under its public key. It is not a record of any real session or customer.
Check the sample in this browser
The sample has 19 entries: tools pinned when first seen, an allowed read, a denied write, a call held until it was approved, a kill-switch drill, and a tool whose definition changed after it was pinned. The second button rewrites the denied write as allowed, in memory, and checks again, so you can see a failure.
Nothing checked yet. Nothing leaves your browser when you do.
Check your own files
Your files are read by this page and never uploaded.
Add a signed tree head or bundle you kept from earlier if you have one. Without it, entries removed from the end of both the ledger and its signatures cannot be noticed.
Check it offline in about a minute
Paste this into a terminal. It downloads the kit, confirms every file against
SHA256SUMS, and runs both checkers and the golden vectors. Expect
VERIFIED twice and self-test: all checks passed.
base=https://deepsweep.ai/evidence-kit/v1
mkdir -p deepsweep-kit/sample && cd deepsweep-kit
for f in verify_ledger.py verify_bundle.py evidence-vectors.v1.json PROVENANCE.json SHA256SUMS \
sample/ledger.jsonl sample/ledger-sig.jsonl sample/ledger.pub.pem \
sample/ledger-head.json sample/evidence-bundle.json; do curl -fsSL -o "$f" "$base/$f"; done
shasum -a 256 -c SHA256SUMS
python3 verify_ledger.py sample/ledger.jsonl sample/ledger-sig.jsonl sample/ledger.pub.pem --tree-head sample/ledger-head.json
python3 verify_bundle.py sample/evidence-bundle.json sample/ledger.pub.pem
python3 verify_bundle.py --self-test evidence-vectors.v1.json No shasum? Use sha256sum -c SHA256SUMS. The hashes in
SHA256SUMS come from the same place as the files, so matching them
shows the download arrived intact. It does not show the files are honest. For that, read the
two scripts: they are short, use only the Python standard library, and make no network calls.
What a passing check proves
No entry was changed, reordered or taken out of the middle by anyone without the signing key
Each entry’s hash covers its position, its time, its kind and everything in its payload, and the next entry repeats that hash, and each entry is signed. Change any of those values, move an entry, or remove one from the middle without the signing key, and the check fails and names the kind of change it found. Whoever holds the key can rewrite and re-sign everything, as described below. The hash is taken over canonical JSON, so spacing, key order and number formatting that leave every value the same are not changes and are not reported. Entries removed from the end of both files leave fewer entries, which only a tree head or bundle signed earlier can show.
Every entry was signed by the pinned key
A pass needs every entry to carry its own Ed25519 signature that verifies under the key you pinned. An entry signed by a different key, or not signed at all, is refused, and a ledger with no signatures is reported as unsigned, never as verified.
Against a head you received earlier, nothing it covers was cut off or rewritten
Checked against a signed tree head you received earlier, a ledger shorter than the head is refused, and its first entries must reproduce the head’s Merkle root, so a later rewrite of those entries is caught, even one made by the key holder. Checked against a signed bundle, only a ledger with fewer entries than the bundle has records is refused, because a bundle is compared by count: its records are not the ledger’s bytes, so a rewrite that keeps the same number of entries passes against it. Both checks run in this browser and in verify_ledger.py.
Each exported record is in the signed tree
The bundle check rebuilds the Merkle root from the records and re-checks every proof against the signed root.
What it does not prove
Not that the entries are true
A record of a denied call is kept as faithfully as a record of an allowed one. The check shows the record was not changed after it was signed by anyone without the signing key, not that it was accurate when written.
Not that the key holder left it unchanged
Whoever holds the signing key can rewrite and re-sign the whole ledger and it will pass; only a tree head you received before the rewrite, or one with an independent timestamp (checked separately; this kit does not check timestamps), catches that; a bundle received earlier catches it only if the rewrite left fewer entries, because a bundle is compared by count. The key holder is usually the operator whose agents the ledger describes.
Not that every action was recorded
An action that was never written to the ledger leaves no trace, so no check can notice it is missing. The ledger covers only what passed through the component that writes it, and that component writes nothing while its ledger file is damaged, for calls the kill switch or a changed tool definition refuses, or when the kill switch, the tool pins or the approvals are changed by editing their files instead of with the gateway’s commands.
Not who ran it, or who holds the key
A signature shows which key signed. Linking that key to a person or a company is a separate step: get the public key from its owner through a channel you trust, not from the same place as the files.
Not when it happened
Timestamps in entries and tree heads are the signer’s own claim. An independent timestamp on a tree head is what bounds when it existed.
Not a cut-off end, without a head or bundle
If the last entries are removed from both the ledger and its signatures, the shorter ledger still checks out on its own, in this browser and offline. Keep a signed tree head or bundle from earlier and check against it.
Not secrecy of short values
Entries hold SHA-256 digests instead of tool names, arguments and results. A digest of a short, guessable value, like a tool name, confirms a guess for anyone who has the file. Treat digests as commitments, not encryption. Some fields are in clear text: fixed codes, counts, the names of your rules, the identifiers of installed policy packs and the categories a call was classified into, the label an operator gives when turning the kill switch on or off or running a drill, and, in entries and bundles written by earlier versions, the workspace folder’s name.
The files
Everything is served from /evidence-kit/v1/. The two scripts carry an Apache-2.0 licence header.
| File | What it is | SHA-256 |
|---|---|---|
| evidence-vectors.v1.json | Golden vectors: known inputs and the hashes and proofs every implementation must reproduce. | 5d8ff21e24a320178e1045d7df682f3843157d46ebd3f2c6a10aa53095541e77 |
| PROVENANCE.json | Where each file came from, its licence, and the sample key id. | 23709b410501b02b2cbb3452b4f13c00c17cecc51e16bc5bf98892eb66a5809f |
| sample/evidence-bundle.json | The sample exported as a signed evidence bundle with Merkle proofs. | 9cbebf02d75ad1000f016b8862d7ef7a241b35276bc8bea63f9e0493352e1268 |
| sample/ledger-head.json | A signed tree head over the whole sample ledger. | 8ad4fb61224497bb6314d54c97e15a14aa6b37a3d43359242aa7f4acfe6ab947 |
| sample/ledger-sig.jsonl | One Ed25519 signature per ledger entry. | eb314fce4a99295c5601711b9bc52f28eec2569c364453a8ac8f3f0ac3184ad8 |
| sample/ledger.jsonl | Sample ledger: 19 entries, one per line. | e0c280764b52b88561bb8b16992b39e9db429fed6fd9d0785639f7c570c261bb |
| sample/ledger.pub.pem | The public key to pin when you check the sample. | 7cdc019e9ebb528317cc3d36d0a48c92155c3d1f7db838aa8a4c99d6b41d9d0e |
| verify_bundle.py | Checks an evidence bundle: every record against the signed Merkle root, and every inclusion and consistency proof. Also runs the golden vectors. | bcd3068aa0217bbef2ad3251cd9bd5e8193815d18d6b0b6fca295a5fbdc52315 |
| verify_ledger.py | Checks a ledger: the hash chain, every entry signature, and optionally a signed tree head or bundle. Python 3.8 or later, standard library only. | 98896c70bd771466229d737fe88476045cb48e2275ae9ab80b18b38e00aaff6d |
The format
This is the part of the DeepSweep Agent Evidence Format the kit checks. It is enough to write a third checker from, which is the point: a format that only its author can check is a claim, not a format.
Primitives
- Canonical JSON: RFC 8785 (JCS): keys sorted, no whitespace, integers as plain numbers.
- Digest: lowercase hex SHA-256 of the UTF-8 bytes.
- Signature: Ed25519 (RFC 8032), base64. Public keys are SPKI, usually PEM.
- Key id:
dsk_followed by the first 16 hex characters of the SHA-256 of the base64 SPKI key.
Ledger entry (one JSON object per line of ledger.jsonl)
{ seq, prevHash, occurredAt, kind, payload, entryHash } - An entry has exactly these six fields; a line with any other field is refused.
entryHashis the SHA-256 of the canonical JSON of the other five. prevHashis the previous entry'sentryHash; the first entry uses 64 zeros.seqstarts at zero and goes up by one, with no gaps.payloadholds digests, fixed codes, counts, the names of the rules you wrote, the identifiers of installed policy packs and the categories a call was classified into, and, on kill-switch entries, the label the operator gave when turning it on or off or running a drill. Review entries written by earlier versions also carry the workspace folder’s name. Beyond that, the format puts no file names, paths, arguments or tool content in a payload; an operator label is whatever the operator typed.
Entry signature (one line per entry of ledger-sig.jsonl)
{ schemaVersion: 1, seq, entryHash, keyId, signature } The signature is Ed25519 over the canonical JSON of
{ schemaVersion: 1, seq, entryHash }. Signing the position with the hash is what
lets a checker tell a reordering apart from an edit. keyId must be the pinned
key's id. Any other field in a signature line is ignored and not signed.
Signed tree head
{ treeHead: { schemaVersion: 1, treeSize, rootHash, signedAt, logId }, signature, keyId } rootHash is an RFC 6962 Merkle root. Leaves are
SHA-256(0x00 || data), inner nodes SHA-256(0x01 || left || right). Over
a ledger, each leaf's data is the 32 bytes of an entryHash; over a bundle, it is the
canonical JSON of a record. logId must equal keyId, and the signature
is Ed25519 over the canonical JSON of treeHead. signedAt is the
signer's claim.
Reading JSON
The in-browser checker and verify_ledger.py read JSON the same strict way, so a
file means the same thing to both. They refuse the non-standard values for not-a-number and
infinity, numbers too large to be finite, integers beyond plus or minus 253 − 1,
text that is not valid Unicode, an object that names the same field twice, a byte-order mark,
and nesting deeper than 64 levels. A tree
size, a proof index and a proof's sizes must be written as integers: 19.0 is
refused there. Canonical JSON follows RFC 8785 including its number format, so inside a
payload 3 and 3.0 are the same value and hash the same.
verify_bundle.py is published unmodified, and reads a bundle more leniently in a
few places: it takes true or false where a number belongs, ignores
stray characters in base64, accepts a digest followed by a line break, keeps the last of two
fields with the same name, and counts inclusion proofs without matching each to its own
record. None of these lets a changed record or signature pass: the signed root is still
rebuilt from every record. But a field named twice can show a person reading the file one
value while the hash covers the other, so treat a bundle this script passes and the in-browser
check refuses as not verified. verify_ledger.py --bundle refuses all of them too.
The order of the ledger checks
A ledger can be wrong in several ways at once. The checks run in this order and the first one to fail names the result.
| Result | When | Shown as |
|---|---|---|
malformed-ledger | A ledger line is not one JSON entry with exactly the six fields, or cannot be read as described under Reading JSON. | Not verified: the ledger file cannot be read |
malformed-signature | The signature file cannot be parsed. | Not verified: the signature file cannot be read |
reordered | Entries are out of sequence but chain correctly once sorted. | Not verified: entries were reordered |
edited | The hash chain breaks. | Not verified: an entry was changed or removed |
truncated | There are more signatures than entries. | Not verified: entries were cut off |
head-refused | A supplied tree head or bundle cannot be read, or fails its own checks. | Not verified: the tree head or bundle was refused |
truncated | A signed tree head or bundle covers more entries than remain. | Not verified: entries were cut off |
edited | The first entries do not reproduce a signed tree head’s root. | Not verified: an entry was changed or removed |
unsigned | The chain is intact and there are no signatures. Never reported as verified. | Consistent, but unsigned |
signature-gap | An entry has no signature. | Not verified: an entry has no signature |
edited | A signature covers a different hash from its entry’s. | Not verified: an entry was changed or removed |
untrusted-key | An entry is signed by a key other than the pinned one. | Not verified: signed by a different key |
forged | A signature does not verify. | Not verified: a signature does not match its entry |
verified | None of the above. | Verified |
Evidence record and bundle
An evidence bundle turns each ledger entry into one record, builds a Merkle tree over the records, and carries an inclusion proof for every record, optional consistency proofs, and a signed tree head. Record fields:
schemaVersion | 2. Records written before version 2 carry 1 and have no review or eventKind field. |
occurredAt | The time the recorder wrote for the entry. |
workspace | A label the organization chose, or a salted digest. Records exported by earlier versions can carry the workspace folder’s name instead. |
outcome | allow, deny, require-approval or observe. |
reason | rule-matched, default-effect, policy-refused, review-run or ledger-event. |
principalHash, actionHash, resourceHash | SHA-256 digests, never the values themselves. |
ruleName | On decision records: the name of the rule that decided, or the fixed text (none — defaultEffect) when the policy’s default decided. Other records carry no rule name. |
policyMode | enforce or observe. |
policyLayers, matchedRuleCount, review | Policy detail, and review counts on review records. |
eventKind | On ledger-event records only: the entry kind, such as gateway.tool_call. |
Bundle fields the checker does not protect
The signature covers the tree head, and through its root every record. It does not cover the bundle's outer labels. Each item below was found by changing one field at a time and seeing the check still pass; treat these as unverified notes, not evidence.
schemaVersion: The bundle's own version number is a label. Nothing checks it.workspace: The workspace name at the top of the bundle is a label. The copy inside each record is covered by the signature; this one is not.generatedAt: The export time at the top of the bundle is a label. The signed tree head carries its own time, which is covered by the signature but is still the signer's claim.treeSize: The record count at the top of the bundle is a label. The count that matters is inside the signed tree head, and the verifier checks that one.inclusion[].selfVerified: Each proof's selfVerified flag is the exporter grading its own work. The verifier ignores it and re-checks every proof itself.consistency[].selfVerified: Same as above, for consistency proofs.inclusion[].treeSize: The tree size written inside an inclusion proof is not compared with the signed tree size, so a changed value can still pass when the proof's shape happens to match. The proof still has to reproduce the signed root, so it cannot place a record that is not there.consistency[].secondSize: Same as above, for the larger tree size inside a consistency proof.delete schemaVersion: Removing the version label is not noticed.delete workspace: Removing the top-level workspace label is not noticed.delete generatedAt: Removing the export time label is not noticed.delete treeSize: Removing the top-level record count is not noticed.delete consistency: Consistency proofs are optional. Removing them removes that check; it does not make the bundle fail.
Two fields are required but not signed: status must be ok and
chainIntact must be true. Anyone assembling a bundle can write both,
so they show only that the exporter did not report a broken chain.
Next
To see where this evidence helps with the NIST AI RMF and the OWASP Top 10 for Agentic Applications, and where it does not, read the framework mapping. For our own published log, checked live, see evidence.