Agent harnesses blindly trust MCP servers' self-reported safety annotations to bypass human approval. This creates a vulnerable trust boundary where tools can lie about their destructive capabilities without behavioral verification.
- Behavioral Audits: Use Airlock to run capped probes that compare a server's claimed safety against actual observed execution.
- Strict Classification: Treat missing sensor coverage explicitly as untested rather than collapsing unobserved execution into a false passing score.
- Wire Enforcement: Deploy an enforcing proxy for HTTP targets that strictly blocks unapproved tool calls before they reach the upstream server.
Black-box probing misses stateful or conditional side effects, and local stdio targets currently lack proxy enforcement, requiring external OS-level sandboxing like Landlock or seccomp.
Script
If your production stack includes MCP servers, and your agent harness skips human approval whenever it sees readOnlyHint: true, you need to hear this. Everyone else can wait for tomorrow.
MCP servers describe their own tools. A tool publishes its own annotations. readOnlyHint, destructiveHint, idempotentHint. Your harness reads those flags and decides whether to show the operator an approval card. That creates a trust boundary where the server being evaluated is also the server describing how dangerous it is. That is not enforcement. That is a promise, and promises can lie.
Introducing Airlock
Himanshu Kumar built Airlock to stop trusting those promises and start recording behavior. It opens a case, inventories a server's declared tools, probes them under a capped budget, and compares what the server claimed against what actually happened.
How Airlock Reports Findings
The output is not a safety score. Airlock deliberately avoids a single aggregate because an average can hide the one tool that lied. Instead, each check resolves to one of four states:
findingno_finding_observednot_testedsensor_failed
Treating not_tested as a first-class outcome is the sharp part. When Airlock only has transcript visibility and cannot see server-side filesystem or network activity, it records the question as untested rather than clean. It even distinguishes capability_absent, where no sensor existed, from evidence_missing, where the sensor could answer but observed nothing. Most evaluation harnesses collapse missing sensor coverage into a passing score. That turns unobserved execution into false confidence.
Proving the Detectors
To prove the detectors work, Airlock ships two six-tool fixtures with identical surfaces. The dishonest fixture plants five behaviors:
- a filesystem write behind a declared read-only hint
- scope escape
- undeclared egress
- injected instructions
- canary exfiltration
Across twenty-four probes and thirty-six checks, Airlock reports seven findings and catches all five planted behaviors. The honest fixture produces zero findings across the same checks. That contrast matters. A detector that only finds problems in suspicious-looking fixtures has not proven much. The control shows the findings follow observed behavior rather than unfamiliarity.
Real-World Audits
On a real deployed target, ContextFirewall, Airlock found six tools, thirty probes, and zero annotations. The server did not lie. It said nothing. A harness that resolves categories like @write or @destructive from annotations can match nothing either way. Tools named remember or forget_memory may skip the approval path entirely because there is no declaration to trigger it.
Airlock also audited five public stdio servers:
server-filesystemwith fourteen toolsserver-everythingwith thirteenmcp-server-gitwith twelveserver-memorywith nineserver-sequential-thinkingwith one
Stdio support matters because many MCP servers launch as local commands rather than HTTP endpoints.
How Airlock Works
Airlock itself runs as an MCP server with six control tools: open_case, list_declared_tools, probe_tool, read_evidence, seal_case, and emit_policy. Three of those are always approval-gated: probe_tool, seal_case, and emit_policy.
An agent drives the audit, reads the aggregate evidence, and presents a human choice: block, approve selected, or approve all. For an allowed case, the emitted connector points to a per-case enforcing proxy rather than back to the suspect server. A call to a tool the case did not approve receives a hard MCP error: tool blocked by Airlock policy. The policy is enforced on the wire before the call reaches the upstream tool. It is not a sentence in a system prompt.
Security and Self-Auditing
Running stdio targets means executing code Airlock exists to distrust. The command never comes from a case argument, tool result, or model-generated string. The operator configures a fixed map of names to argument arrays. A case selects a name, and names are looked up, never parsed into shell commands. The child starts in a throwaway working directory with an explicit environment. Airlock revalidates the whole command binding before each connection, so repointing a configured name revokes an open case instead of silently running a different binary.
Airlock also documents a remaining boundary: its HTTP response cap does not apply to the MCP SDK's stdio transport. A large line can reach the SDK before Airlock can bound it. Calling that solved would be worse than stating it. Even the self-audit failed, and Kumar published the failure. Airlock's own open_case control tool could not be fully probed because its schema contained a $ref into $defs, outside the bounded v1 probe profile. That case is marked incomplete on the public page, because a product arguing that missing evidence is not proof of safety should not hide its own missing evidence.
Qodo's automated reviews found real bugs in Airlock's own code. On the stdio transport, revalidation originally compared only the target name, meaning repointing that name could leave an open case running an operator-withdrawn command. The MCP SDK also inherited more host environment variables than the README claimed. On the approval boundary, a test claimed to cover every side-effecting control tool while checking three hard-coded names. The first fix introduced a second map that could drift, and Qodo caught that too. The decorators now read from the same policy map the test validates.
Another review found that a passing-suite claim did not survive a clean CI machine. The failures were fixed, re-reviewed, and recorded. The current suite contains three hundred and two passing tests. Those reviews caught statements that were stronger than the observed behavior, which is exactly the failure Airlock is designed to expose.
Limitations
Now, the limits. A tool that only writes on its Nth invocation, or under a specific payload, will likely evade black-box probing. Airlock starts a fresh stdio process per probe, so cross-call state is invisible. An approved tool carrying an unobserved conditional side effect still reaches the wire. The author acknowledges this openly: it is a structural limit of black-box probing, not a tuning problem.
Format and Packaging Issues
There is a format problem too. MCPB bundle manifests carry no tool annotations. They only store name and description, plus a tools_generated flag. A missing declaration on a bundle is not necessarily operator dishonesty; the format cannot express intent. The running server could still annotate at runtime, but Airlock cannot yet distinguish why annotations are absent: author omission, build-generated catalog, or packaging loss.
And if tools_generated is true, the catalog can expand at runtime. Airlock pins a digest and fails closed on drift, which means legitimate expansions will also break the case. That flag is self-reported and defaults to false, so it cannot become an input to policy without reintroducing the exact trust Airlock rejects.
Stdio Target Boundaries
For stdio targets, Airlock is explicit about its boundary. It isolates the working directory and environment, but does not claim to sandbox filesystem or network calls. That containment belongs to your deployment. Additionally, emit_policy refuses to create a connector for stdio targets because there is no HTTP upstream to hold the policy. Stdio is audit-only today.
Who Is This For?
Prototype Airlock next week if you run HTTP-based MCP servers with agent harnesses that gate approval on self-reported annotations. It gives you behavioral evidence, an enforcing proxy, and a workflow that treats missing coverage as uncertainty instead of a pass.
Bookmark it and wait if your tools rely on stateful sequences, conditional side effects, or MCPB bundle distribution. The next layer needs static capability indicators, stateful sequence probes, and OS-level deny-by-default containment like Landlock or seccomp. Those are acknowledged future work, not current features.
Actively avoid it if you need a compliance checkbox. Airlock is open source under the MIT license, built for a hackathon, and moving fast. It will not give you a green safety score. It will give you a case file with explicit gaps, and it will force you to look at them. That is the entire point.
This is TAKEYOURPILLS.TECH. Go ship something.