When the middleman pads the meter
Many organisations reach AI models through third-party relay services, and an audit of commercial relays found some altering answers, adding unrequested tool use and over-billing. Agent traffic should be signed end to end, usage should be checked independently before paying, and tools should run where a relay can't change their permissions.
Instead of contracting with each AI provider, many teams buy access through a relay: one account, one bill, many models. The relay sits between your agent and the model and can see, alter and bill every request. If the only record of usage comes from the relay, you can't check it.
The rise of the convenient middleman
When an organization begins building autonomous assistants, managing dozens of frontier model accounts gets cumbersome. Every provider uses different credentials, distinct rate limits, and separate billing dashboards.
To simplify this plumbing, many engineering teams turn to third-party API relays. These middleman services sit between your internal application and the model providers. You send all your requests through a single URL. The relay handles routing, balances traffic, converts formats, and sometimes offers wholesale discounts.
On paper, this arrangement sounds painless. You pay one consolidated bill at the end of the month. Your code stays neat, and your team avoids dealing with multiple vendor contracts.
However, inserting an unmonitored proxy into your pipeline surrenders control. You trust the middleman to deliver your prompt untouched, call the exact model you requested, and charge only for what you consumed.
What this means
An API relay is an active party with its own financial incentives, capable of altering prompts, permissions, and billing records without your knowledge.
What empirical audits uncovered
Until recently, teams had to take the honesty of these relays on faith. That changed with a rigorous empirical investigation published by computer scientists in November 2026.1
The researchers audited twenty commercial API relays providing access to leading models. They sent thousands of identical requests through both official endpoints and commercial relays, comparing the results byte by byte.
The findings showed that unmonitored middlemen frequently break their promises in three distinct ways:
First, several relays inflated billed usage. Five commercial services added phantom input tokens to identical requests, charging customers for thousands of tokens that were never sent. In the most extreme instances, relays reported up to sixty thousand extra tokens on standard tasks. That is the digital equivalent of a taxi driver spinning the dashboard meter by hand.
Second, multiple relays silently altered permissions. When researchers explicitly told the model not to execute external tools, four relays disregarded the instruction. They forwarded responses containing unauthorized tool invocations. If an enterprise agent connects to live internal databases or email systems, an unmonitored middleman can breach your security perimeter.
Third, answers differed. Most of the low-price relays tested gave answers that differed measurably from the official model's, so customers couldn't be sure which model they were paying for. Retesting changed some of those results, which is itself a reason to measure repeatedly.
Why network boundaries fail inside the pipeline
Traditional IT security focuses on defending the outside walls of the company. Firewalls check incoming web traffic, and passwords guard company accounts.
Once an AI assistant operates inside those walls, traditional perimeters crumble. When an assistant asks a database for customer records, the request comes from an authenticated internal employee token. Traditional filters see nothing suspicious.
Recent enterprise security reporting highlights that data lineage has become the true security perimeter.2 You cannot evaluate safety merely by checking where a packet originated. You must know who drafted the prompt, which tools were permitted, and whether the text was modified along the way.
When you pass data through an unverified relay, that chain of evidence vanishes. You hand plaintext company queries to an outside vendor who may log prompts, train internal models on your data, or inject subtle prompts.
Three practical rules for builders
Securing your agent architecture does not require abandoning third-party routing. It requires treating every middleman as an untrusted carrier, following established zero-trust principles.3
Here are three concrete engineering defenses:
First, count tokens locally before sending. Never rely on the billing number reported by an outside proxy. Run an offline tokenizer on your client machine before transmitting the request. If the relay bills you for ten thousand tokens when your local counter registered four thousand, flag the discrepancy immediately and pause automated payments.
Second, sign your messages end-to-end. Modern internet standards enable cryptographic request signatures.4 By signing your prompt and parameter headers at the client, you ensure that any alteration made by an intermediate relay will invalidate the signature at the provider.
Third, enforce a strict tool airlock. Autonomous assistants must never trust tool calls returned directly by a proxy. Instead, validate every tool invocation against a local allowlist. If your application disabled tool execution for a specific query, discard any tool payload returned by the network. Do not let a middleman decide which functions your code may run.
Honest limits
Client-side verification stops meter padding and catches altered permissions. But it does not solve every vulnerability.
If a frontier model provider suffers a core outage, client checks cannot revive it. If a malicious insider with legitimate keys alters code inside your repository, signatures cannot detect the betrayal. Maintaining local tokenizers across dozens of open and proprietary model architectures also introduces modest maintenance overhead for developer teams.
Technology alone cannot replace sound vendor governance. Teams must still review privacy terms, maintain direct relationships with foundational labs, and periodically audit their network traffic.
By verifying what middlemen deliver rather than trusting advertised labels, we turn fragile plumbing into robust engineering. You can inspect tested audit steps and timeline encounters in our Shared Horizon action library.
Related essays: Paying for words instead of the repair · AI agents should hold permissions that expire · Why an AI agent will not audit itself. Further reading: Why an AI agent will not audit itself · Postmarks for the real world.
- X. Wu, Z. Zhao, Q. Li, C. Cheong, X. Li, and J. Ni, “What the Middleman Delivers: Answer Behavior, Tool Permissions, and Billed Usage in LLM API Relays,” arXiv preprint arXiv:2611.07650 (2026).
- MIT Technology Review Insights, “Securing the agentic enterprise with data lineage,” MIT Technology Review (November 4, 2026).
- S. Rose, O. Borchert, S. Mitchell, and S. Connelly, “Zero Trust Architecture,” NIST Special Publication 800-207, National Institute of Standards and Technology (2020).
- A. Backman, J. Richer, and M. Sporny, “HTTP Message Signatures,” RFC 9421, Internet Engineering Task Force (September 2023).
