AI gateway triple-gate:
LLM, MCP, and A2A traffic under one policy plane

Why model-only gateways lie about control

Early AI gateways solved a real problem: one API key, many models, failover, and basic logging. That design assumed the risky action was the completion. Agentic systems broke the assumption.

Today the completion may only decide which tool to call. The money move, ticket update, or peer agent handoff happens on MCP or agent-to-agent channels. A gateway that never sees those channels reports "healthy LLM traffic" while production side effects go ungoverned.

Control means seeing every action class that can spend money, move data, or change state. Completions are only one class. For MCP specifics, see MCP enterprise gateways.

Operator scenario: the LLM gate shows normal token volume on a quiet afternoon. Downstream, an agent has opened twenty write tools and handed work to a peer that holds broader CRM scopes. Finance notices the CRM API bill first. The model dashboard never blinked.

Failure modes when gates are partial

Shadow tool paths: SDKs call MCP servers directly, bypassing the model gateway's policies. Developers did it once for latency in a spike, then left the shortcut in the agent image.

A2A privilege bounce: Agent A cannot call a tool, so it asks Agent B, which can. Policy on A is irrelevant. Without A2A allowlists and payload limits, every privileged agent becomes a confused deputy for its peers.

Cost blind spots: token dashboards look fine while tool fan-out and secondary agents burn the real budget. Workflow owners cannot answer "what did this feature cost last week?"

Incident fog: security gets a chat transcript without tool IDs or handoff spans, so root cause stalls. Teams argue about the prompt while the damaging action was a peer message nobody logged.

Split kill switches: ops pauses the model provider during a bad release, but MCP and A2A keep running on cached plans. Half the system freezes; the other half keeps writing.

Policy drift across SDKs: one team uses the gateway client; another uses a raw provider SDK plus a direct MCP URL. Identical product features inherit different controls based on who wrote the harness.

The triple-gate architecture

Unify identity at the edge. Every LLM call, MCP call, and A2A message carries a principal, a run ID, and a task or role claim. The same claims drive allowlists across classes so a read-only task cannot escalate by switching channels.

Apply policy per traffic class. LLM routes choose models and prompts. MCP routes choose tools and schemas. A2A routes choose which agents may talk and what payloads they may exchange. Shared fields: tenant, environment, budget pool, and sensitivity tier.

Share budgets and kill switches. Pausing "the model" is not enough if tools and peer agents keep running. Independent switches per class prevent half-frozen systems. Prefer switches that fail closed for writes and fail soft for reads when that matches risk.

Operable build order: terminate TLS and auth at the edge, stamp run IDs, enforce allowlists per class, emit correlated spans, then wire budgets and kill switches. Do not ship fancy routing before identity and deny-by-default work.

Policy, routing, and cost under one plane

Classify tasks: FAQ, internal research, customer write, irreversible change. Route cheap models to low-stakes work and reserve frontier models for hard reasoning. Details live in cost-aware model routing.

Attach spend and rate limits to teams and run types. Attribute cost to the business workflow, not only to the model vendor invoice line. Include tool call counts and peer agent minutes in the same budget object so "cheap model, expensive tools" still trips the limit.

Reject unknown MCP tools and unknown A2A peers by default. Discovery is useful in labs; production needs an allowlist with owners. Require schema versions on MCP and payload schemas on A2A. Unsigned or unversioned traffic is a deny.

Encode dual-control for irreversible A2A and MCP routes: the gateway checks that the task claim, the tool class, and any human gate span all agree before forward. A peer request without the original user's entitlement should fail even if the peer agent is highly privileged for its own work.

Verification before you widen autonomy

Eval suites should include bypass attempts: direct tool calls, peer escalation, and budget exhaustion. Gate gateway releases on those cases. Include "Agent A asks Agent B to do what A cannot" as a first-class security case, not a curiosity.

Correlate traces: one run ID across LLM spans, tool spans, and handoffs. If you cannot draw that graph, observability is incomplete (agent observability). Sample production runs weekly and confirm span completeness as a metric, not a hope.

Tabletop an incident: disable a bad tool, freeze an agent role, and fail over a model provider without restarting the whole platform. Time each action. If any step needs a code deploy, the kill switch is not real yet.

Chaos-test shadow paths: deploy a canary agent that tries a direct MCP URL and confirm detection and deny. Undetected bypasses are how triple-gate designs rot.

Rollout and metrics that matter

Adopt the gateway for new agent workloads first, then migrate shadow paths. Do not declare victory while any write tool still has a direct credential in an agent env. Inventory SDKs and env secrets as part of the migration checklist.

Measure policy deny rates, bypass detections, cost per workflow, span completeness, and mean time to pause a traffic class. Those numbers beat vanity "requests served" charts. Track the percentage of MCP and A2A traffic that still bypasses the edge; treat non-zero as a sev-worthy backlog for write paths.

Document who owns each gate rule and how teams request new routes. A triple-gate without ownership becomes a shared black box that every team tries to route around. Publish a short runbook: how to add a model route, how to register a tool, how to allow a peer pair, and how to revoke any of them under pressure.