Production readiness for agentic AI
Find out what your agent can actually do, before your review board does.
PRIVI compares what your team says an agent is allowed to do against what its code actually does, and reports every difference with the file and line that proves it.
The checks are published in full, with a content hash, so a result cites a specific text rather than a version number. Where the tooling cannot settle something, it says so instead of scoring it.
- Every check published before any assessment runs
- Every finding carries the file and line it was read from
- What the code does beats what the declaration claims
- Re-runs over the same input produce the same bytes
- 21 of the 41 checks are not automatable, and the standard says which
Start an assessment See a complete example → Read all 41 checks → What we measured
What you get, and keep.
Five artifacts, all of them checkable by somebody who was not in the room.
- A score against every check, with the evidence each one rests on and where in the codebase it is.
- A written statement of what your agent is allowed to do — every tool, permission and approval rule, in one vendor-neutral file. (The tooling calls it an Agent Contract.) It is yours either way.
- A remediation plan, ordered by blast radius rather than by effort.
- The harness, so your team re-runs the automated part on every change without us.
- A per-release answer: which approvals a change stopped being valid, and which still hold.
Declare · Observe · Reconcile · Decide · Re-check
The same five steps on every engagement, in the same order, with an artifact at each one.
- DeclareWhat the agent is permitted to do is written down once, as data, by the team accountable for it.
- ObserveThe code is parsed. Every tool, model, permission and approval rule the implementation actually carries is recorded, with its file and line.
- ReconcileThe two are compared. Anything the code carries that the declaration does not is a finding — and so is anything neither can settle, which is reported rather than scored.
- DecideA verdict from named rules, each one printed in the artifact, with the exact conditions that would clear it. No score anywhere in the path.
- Re-checkThe next release re-runs it and reports only what moved: which approvals it cost, and which survived by name.
One real comparison, every part of it, nothing selected for effect.
This is the complete result the tooling produces on an agent whose declaration and implementation disagree. Every file behind it is published beside this page. Nothing below is a mock-up.
1 · What the team declared
acme-records-assistant v2.3.0 — the written statement of what this agent is allowed to do.
Answers questions about the records register by reading it. Declared read-only: it holds no write scope, takes no action that changes a record, and has no way to transmit anything outside the platform.
| Tool it declares | What it may do |
|---|---|
lookup-account | read |
search-records | read |
2 · What the code actually registers
Found by parsing the TypeScript. This tool is written down nowhere as data — there is no manifest entry to read, so anything that only reads manifests never sees it.
src/tools/registry.ts line 69
| 69 | export const postToPartnerWebhook = defineTool({ |
| 70 | name: 'post_to_partner_webhook', |
| 71 | description: 'Send the record payload to the partner integration endpoint.', |
| 72 | inputSchema: WebhookPayloadSchema, |
| 73 | sideEffect: 'send', |
| 74 | requiresApproval: false, |
| 75 | timeoutMs: 15000, |
| 76 | maxRetries: 5, |
| 77 | scopes: ['webhook.post', 'records.read'], |
| 78 | }); |
It sends. It asks nobody first. It is not in the declaration above, which means no approval stands in front of it and no test points at it.
3 · What the check returns
BLOCK A change of critical severity widens what the agent may do.
10 of the 29 approvals, tests and sign-offs recorded for this agent stopped being valid. The other 19 still hold, and the tooling says which is which rather than asking anyone to re-check everything.
Read the whole comparison, with every file it was computed from →
- One agent, two versions of its declaration — five named changes, and the cost of each.
- Evidence produced by running the thing — the same agent hardened and unsafe, and what a local test genuinely cannot establish.
- A full sample report — the automated subset against a deliberately unsafe agent.
Almost no agent can answer the first question.
We pointed the same instruments at other people's code. Both figures carry the total they were counted out of, and both link to the file they came from.
2.3%103 of 4,445
Tools found by reading 200 public agent codebases. Only these say whether they change anything — write, send, spend or delete. The other 4,342 say nothing either way.
20,451servers, whole registry
Entries in the largest public registry of agent declarations, counted until the listing ran out. Every one records which package to run and where to reach it. The registry has no field for what the agent may do — not one usually left empty; there is nowhere to put it.
What this does not do.
- It does not certify anything. The standard says so in its own scoring model, and no score produced under it is a certification.
- 21 of the 41 checks cannot be read from code at all. They need a running system, or interviews and document review. The standard publishes which, rather than letting a scanner imply it settled them.
- A declaration that is simply wrong about something no parser can see is not caught by this, or by anything else here.
- It is not a judgement about whether the agent is good at its job. That is a capability question; this is a readiness one, and conflating them is how a security review gets answered with a demo.
- No second party has applied this standard without us. Until that happens it is a rubric with a version history rather than a standard, and it carries that sentence rather than burying it.
An agent you cannot describe is an agent you cannot approve — and cannot re-approve after it changes.