Evidence Verification Kit

Check the evidence yourself

A DeepSweep evidence ledger is a hash-chained record of the decisions made on an AI agent's tool calls: allowed, denied, or held for approval. Each entry can also carry its own signature, when the gateway that writes it is given a signing key. This kit lets you check a ledger without trusting us: in this page, or offline with two short Python files and nothing to install. It also says, as plainly as what a pass proves, what a pass does not prove.

What is available today

The kit is. The format, both checkers, the golden vectors and a signed sample ledger are published here, and the checkers run on any machine with Python 3.8 or later.

The component that writes this ledger in real use is not yet generally available. The sample below is synthetic: an invented agent session written in the same format, signed with a key created for it and then discarded, so nobody, us included, can add an entry that checks out under its public key. It is not a record of any real session or customer.

Check the sample in this browser

The sample has 19 entries: tools pinned when first seen, an allowed read, a denied write, a call held until it was approved, a kill-switch drill, and a tool whose definition changed after it was pinned. The second button rewrites the denied write as allowed, in memory, and checks again, so you can see a failure.

Nothing checked yet. Nothing leaves your browser when you do.

Check your own files

Your files are read by this page and never uploaded.

Add a signed tree head or bundle you kept from earlier if you have one. Without it, entries removed from the end of both the ledger and its signatures cannot be noticed.

Check it offline in about a minute

Paste this into a terminal. It downloads the kit, confirms every file against SHA256SUMS, and runs both checkers and the golden vectors. Expect VERIFIED twice and self-test: all checks passed.

base=https://deepsweep.ai/evidence-kit/v1
mkdir -p deepsweep-kit/sample && cd deepsweep-kit
for f in verify_ledger.py verify_bundle.py evidence-vectors.v1.json PROVENANCE.json SHA256SUMS \
  sample/ledger.jsonl sample/ledger-sig.jsonl sample/ledger.pub.pem \
  sample/ledger-head.json sample/evidence-bundle.json; do curl -fsSL -o "$f" "$base/$f"; done
shasum -a 256 -c SHA256SUMS
python3 verify_ledger.py sample/ledger.jsonl sample/ledger-sig.jsonl sample/ledger.pub.pem --tree-head sample/ledger-head.json
python3 verify_bundle.py sample/evidence-bundle.json sample/ledger.pub.pem
python3 verify_bundle.py --self-test evidence-vectors.v1.json

No shasum? Use sha256sum -c SHA256SUMS. The hashes in SHA256SUMS come from the same place as the files, so matching them shows the download arrived intact. It does not show the files are honest. For that, read the two scripts: they are short, use only the Python standard library, and make no network calls.

What a passing check proves

No entry was changed, reordered or taken out of the middle by anyone without the signing key

Each entry’s hash covers its position, its time, its kind and everything in its payload, and the next entry repeats that hash, and each entry is signed. Change any of those values, move an entry, or remove one from the middle without the signing key, and the check fails and names the kind of change it found. Whoever holds the key can rewrite and re-sign everything, as described below. The hash is taken over canonical JSON, so spacing, key order and number formatting that leave every value the same are not changes and are not reported. Entries removed from the end of both files leave fewer entries, which only a tree head or bundle signed earlier can show.

Every entry was signed by the pinned key

A pass needs every entry to carry its own Ed25519 signature that verifies under the key you pinned. An entry signed by a different key, or not signed at all, is refused, and a ledger with no signatures is reported as unsigned, never as verified.

Against a head you received earlier, nothing it covers was cut off or rewritten

Checked against a signed tree head you received earlier, a ledger shorter than the head is refused, and its first entries must reproduce the head’s Merkle root, so a later rewrite of those entries is caught, even one made by the key holder. Checked against a signed bundle, only a ledger with fewer entries than the bundle has records is refused, because a bundle is compared by count: its records are not the ledger’s bytes, so a rewrite that keeps the same number of entries passes against it. Both checks run in this browser and in verify_ledger.py.

Each exported record is in the signed tree

The bundle check rebuilds the Merkle root from the records and re-checks every proof against the signed root.

What it does not prove

Not that the entries are true

A record of a denied call is kept as faithfully as a record of an allowed one. The check shows the record was not changed after it was signed by anyone without the signing key, not that it was accurate when written.

Not that the key holder left it unchanged

Whoever holds the signing key can rewrite and re-sign the whole ledger and it will pass; only a tree head you received before the rewrite, or one with an independent timestamp (checked separately; this kit does not check timestamps), catches that; a bundle received earlier catches it only if the rewrite left fewer entries, because a bundle is compared by count. The key holder is usually the operator whose agents the ledger describes.

Not that every action was recorded

An action that was never written to the ledger leaves no trace, so no check can notice it is missing. The ledger covers only what passed through the component that writes it, and that component writes nothing while its ledger file is damaged, for calls the kill switch or a changed tool definition refuses, or when the kill switch, the tool pins or the approvals are changed by editing their files instead of with the gateway’s commands.

Not who ran it, or who holds the key

A signature shows which key signed. Linking that key to a person or a company is a separate step: get the public key from its owner through a channel you trust, not from the same place as the files.

Not when it happened

Timestamps in entries and tree heads are the signer’s own claim. An independent timestamp on a tree head is what bounds when it existed.

Not a cut-off end, without a head or bundle

If the last entries are removed from both the ledger and its signatures, the shorter ledger still checks out on its own, in this browser and offline. Keep a signed tree head or bundle from earlier and check against it.

Not secrecy of short values

Entries hold SHA-256 digests instead of tool names, arguments and results. A digest of a short, guessable value, like a tool name, confirms a guess for anyone who has the file. Treat digests as commitments, not encryption. Some fields are in clear text: fixed codes, counts, the names of your rules, the identifiers of installed policy packs and the categories a call was classified into, the label an operator gives when turning the kill switch on or off or running a drill, and, in entries and bundles written by earlier versions, the workspace folder’s name.

The files

Everything is served from /evidence-kit/v1/. The two scripts carry an Apache-2.0 licence header.

FileWhat it isSHA-256
evidence-vectors.v1.json Golden vectors: known inputs and the hashes and proofs every implementation must reproduce. 5d8ff21e24a320178e1045d7df682f3843157d46ebd3f2c6a10aa53095541e77
PROVENANCE.json Where each file came from, its licence, and the sample key id. 23709b410501b02b2cbb3452b4f13c00c17cecc51e16bc5bf98892eb66a5809f
sample/evidence-bundle.json The sample exported as a signed evidence bundle with Merkle proofs. 9cbebf02d75ad1000f016b8862d7ef7a241b35276bc8bea63f9e0493352e1268
sample/ledger-head.json A signed tree head over the whole sample ledger. 8ad4fb61224497bb6314d54c97e15a14aa6b37a3d43359242aa7f4acfe6ab947
sample/ledger-sig.jsonl One Ed25519 signature per ledger entry. eb314fce4a99295c5601711b9bc52f28eec2569c364453a8ac8f3f0ac3184ad8
sample/ledger.jsonl Sample ledger: 19 entries, one per line. e0c280764b52b88561bb8b16992b39e9db429fed6fd9d0785639f7c570c261bb
sample/ledger.pub.pem The public key to pin when you check the sample. 7cdc019e9ebb528317cc3d36d0a48c92155c3d1f7db838aa8a4c99d6b41d9d0e
verify_bundle.py Checks an evidence bundle: every record against the signed Merkle root, and every inclusion and consistency proof. Also runs the golden vectors. bcd3068aa0217bbef2ad3251cd9bd5e8193815d18d6b0b6fca295a5fbdc52315
verify_ledger.py Checks a ledger: the hash chain, every entry signature, and optionally a signed tree head or bundle. Python 3.8 or later, standard library only. 98896c70bd771466229d737fe88476045cb48e2275ae9ab80b18b38e00aaff6d

The format

This is the part of the DeepSweep Agent Evidence Format the kit checks. It is enough to write a third checker from, which is the point: a format that only its author can check is a claim, not a format.

Primitives

  • Canonical JSON: RFC 8785 (JCS): keys sorted, no whitespace, integers as plain numbers.
  • Digest: lowercase hex SHA-256 of the UTF-8 bytes.
  • Signature: Ed25519 (RFC 8032), base64. Public keys are SPKI, usually PEM.
  • Key id: dsk_ followed by the first 16 hex characters of the SHA-256 of the base64 SPKI key.

Ledger entry (one JSON object per line of ledger.jsonl)

{ seq, prevHash, occurredAt, kind, payload, entryHash }
  • An entry has exactly these six fields; a line with any other field is refused. entryHash is the SHA-256 of the canonical JSON of the other five.
  • prevHash is the previous entry's entryHash; the first entry uses 64 zeros. seq starts at zero and goes up by one, with no gaps.
  • payload holds digests, fixed codes, counts, the names of the rules you wrote, the identifiers of installed policy packs and the categories a call was classified into, and, on kill-switch entries, the label the operator gave when turning it on or off or running a drill. Review entries written by earlier versions also carry the workspace folder’s name. Beyond that, the format puts no file names, paths, arguments or tool content in a payload; an operator label is whatever the operator typed.

Entry signature (one line per entry of ledger-sig.jsonl)

{ schemaVersion: 1, seq, entryHash, keyId, signature }

The signature is Ed25519 over the canonical JSON of { schemaVersion: 1, seq, entryHash }. Signing the position with the hash is what lets a checker tell a reordering apart from an edit. keyId must be the pinned key's id. Any other field in a signature line is ignored and not signed.

Signed tree head

{ treeHead: { schemaVersion: 1, treeSize, rootHash, signedAt, logId }, signature, keyId }

rootHash is an RFC 6962 Merkle root. Leaves are SHA-256(0x00 || data), inner nodes SHA-256(0x01 || left || right). Over a ledger, each leaf's data is the 32 bytes of an entryHash; over a bundle, it is the canonical JSON of a record. logId must equal keyId, and the signature is Ed25519 over the canonical JSON of treeHead. signedAt is the signer's claim.

Reading JSON

The in-browser checker and verify_ledger.py read JSON the same strict way, so a file means the same thing to both. They refuse the non-standard values for not-a-number and infinity, numbers too large to be finite, integers beyond plus or minus 253 − 1, text that is not valid Unicode, an object that names the same field twice, a byte-order mark, and nesting deeper than 64 levels. A tree size, a proof index and a proof's sizes must be written as integers: 19.0 is refused there. Canonical JSON follows RFC 8785 including its number format, so inside a payload 3 and 3.0 are the same value and hash the same.

verify_bundle.py is published unmodified, and reads a bundle more leniently in a few places: it takes true or false where a number belongs, ignores stray characters in base64, accepts a digest followed by a line break, keeps the last of two fields with the same name, and counts inclusion proofs without matching each to its own record. None of these lets a changed record or signature pass: the signed root is still rebuilt from every record. But a field named twice can show a person reading the file one value while the hash covers the other, so treat a bundle this script passes and the in-browser check refuses as not verified. verify_ledger.py --bundle refuses all of them too.

The order of the ledger checks

A ledger can be wrong in several ways at once. The checks run in this order and the first one to fail names the result.

ResultWhenShown as
malformed-ledger A ledger line is not one JSON entry with exactly the six fields, or cannot be read as described under Reading JSON. Not verified: the ledger file cannot be read
malformed-signature The signature file cannot be parsed. Not verified: the signature file cannot be read
reordered Entries are out of sequence but chain correctly once sorted. Not verified: entries were reordered
edited The hash chain breaks. Not verified: an entry was changed or removed
truncated There are more signatures than entries. Not verified: entries were cut off
head-refused A supplied tree head or bundle cannot be read, or fails its own checks. Not verified: the tree head or bundle was refused
truncated A signed tree head or bundle covers more entries than remain. Not verified: entries were cut off
edited The first entries do not reproduce a signed tree head’s root. Not verified: an entry was changed or removed
unsigned The chain is intact and there are no signatures. Never reported as verified. Consistent, but unsigned
signature-gap An entry has no signature. Not verified: an entry has no signature
edited A signature covers a different hash from its entry’s. Not verified: an entry was changed or removed
untrusted-key An entry is signed by a key other than the pinned one. Not verified: signed by a different key
forged A signature does not verify. Not verified: a signature does not match its entry
verified None of the above. Verified

Evidence record and bundle

An evidence bundle turns each ledger entry into one record, builds a Merkle tree over the records, and carries an inclusion proof for every record, optional consistency proofs, and a signed tree head. Record fields:

schemaVersion2. Records written before version 2 carry 1 and have no review or eventKind field.
occurredAtThe time the recorder wrote for the entry.
workspaceA label the organization chose, or a salted digest. Records exported by earlier versions can carry the workspace folder’s name instead.
outcomeallow, deny, require-approval or observe.
reasonrule-matched, default-effect, policy-refused, review-run or ledger-event.
principalHash, actionHash, resourceHashSHA-256 digests, never the values themselves.
ruleNameOn decision records: the name of the rule that decided, or the fixed text (none — defaultEffect) when the policy’s default decided. Other records carry no rule name.
policyModeenforce or observe.
policyLayers, matchedRuleCount, reviewPolicy detail, and review counts on review records.
eventKindOn ledger-event records only: the entry kind, such as gateway.tool_call.

Bundle fields the checker does not protect

The signature covers the tree head, and through its root every record. It does not cover the bundle's outer labels. Each item below was found by changing one field at a time and seeing the check still pass; treat these as unverified notes, not evidence.

  • schemaVersion: The bundle's own version number is a label. Nothing checks it.
  • workspace: The workspace name at the top of the bundle is a label. The copy inside each record is covered by the signature; this one is not.
  • generatedAt: The export time at the top of the bundle is a label. The signed tree head carries its own time, which is covered by the signature but is still the signer's claim.
  • treeSize: The record count at the top of the bundle is a label. The count that matters is inside the signed tree head, and the verifier checks that one.
  • inclusion[].selfVerified: Each proof's selfVerified flag is the exporter grading its own work. The verifier ignores it and re-checks every proof itself.
  • consistency[].selfVerified: Same as above, for consistency proofs.
  • inclusion[].treeSize: The tree size written inside an inclusion proof is not compared with the signed tree size, so a changed value can still pass when the proof's shape happens to match. The proof still has to reproduce the signed root, so it cannot place a record that is not there.
  • consistency[].secondSize: Same as above, for the larger tree size inside a consistency proof.
  • delete schemaVersion: Removing the version label is not noticed.
  • delete workspace: Removing the top-level workspace label is not noticed.
  • delete generatedAt: Removing the export time label is not noticed.
  • delete treeSize: Removing the top-level record count is not noticed.
  • delete consistency: Consistency proofs are optional. Removing them removes that check; it does not make the bundle fail.

Two fields are required but not signed: status must be ok and chainIntact must be true. Anyone assembling a bundle can write both, so they show only that the exporter did not report a broken chain.

Next

To see where this evidence helps with the NIST AI RMF and the OWASP Top 10 for Agentic Applications, and where it does not, read the framework mapping. For our own published log, checked live, see evidence.