Updated August 2026. This post originally argued that a gateway is the better default for large MCP tool catalogues. We shipped that design, ran it in production, and measured it — and the recommendation did not survive the data. The analysis of the problem below still holds; the conclusion has changed. The full retrospective is in we built a one-tool MCP gateway, then we deleted it.
When you wire an agent up to a system over MCP, the obvious move is to expose everything. You have 50 operations the agent might want — list projects, read tickets, open tickets, publish contracts, close handoffs — so you register 50 tools. Each one gets a name, a description, and a JSON schema. The agent reads the catalogue and picks what it needs.
This works beautifully at five tools. It starts to wobble at twenty. By fifty it has quietly become one of the biggest reliability and cost problems in your stack — and the failure mode is subtle enough that most teams don't trace it back to the tool surface.
The question is what to do about it. One answer is a gateway: expose one intent-shaped entry point and let a router resolve natural-language intent to the right underlying calls. The other is curation: keep the tools direct and typed, but merge the ones that overlap and name what remains precisely. We tried the first and now recommend the second. Here is the reasoning on both sides, with the costs paid honestly.
The instinct to expose everything is reasonable — at first
Direct tool exposure has real virtues. The schema is the contract: the agent sees exactly what arguments each tool takes and what it returns. There's no extra inference layer to debug, no second model deciding what you meant. Latency is one hop. When something breaks, you can read the tool call and the response and know what happened.
For a small, stable, mostly-read-only surface, this is the correct design. Don't add a gateway to wrap three tools. You'd be paying for machinery you don't need.
The problem is that tool surfaces rarely stay small.
Where direct exposure degrades
Context cost scales with the catalogue, not with the task. Every registered tool's name, description, and schema gets loaded into the model's context — on every request — whether or not the agent uses it. A rich tool with nested arguments and enum constraints can run hundreds of tokens. Fifty of those is a standing tax on every turn, eating context budget that should belong to the actual work. The agent pays to know about tools it will never call this turn.
Routing ambiguity grows faster than the catalogue. With fifty tools, you inevitably have clusters that look alike: get_ticket, get_ticket_thread, get_ticket_activity, get_ticket_audit_log. The agent now has to disambiguate between near-synonyms based on terse descriptions. It picks the plausible-but-wrong one, gets a response that sort of looks right, and proceeds on bad data. More tools means more of these collisions, and they're hard to catch because nothing errors — the agent just quietly does the wrong thing.
Security surface is the whole catalogue, all the time. If every mutating tool is directly callable, every one of them is one confused inference away from firing. A delete_handoff_request sitting next to get_handoff_requests in the catalogue is a live wire. You can add per-tool guards, but now you're enforcing safety in fifty places instead of one.
The gateway alternative, and why it's seductive
A gateway inverts the arrangement. Instead of exposing every operation, you expose a small intent surface — one tool. The agent sends natural-language intent ("close the ticket about the broken login redirect"). A router model interprets it, selects the right underlying tool(s), invokes them, paraphrases what it's about to do, and — for anything that mutates state — confirms before committing. This is the design we shipped as ask_apphandoff.
On paper the wins map directly onto the problems above:
Context stays flat. The agent loads one tool schema, not fifty. The full catalogue lives behind the gateway and is consulted by the router, not carried in the calling agent's context every turn. Adding a 51st tool costs the agent nothing. This part was true in production.
Routing becomes a first-class job. Disambiguating get_ticket from a thread-reading variant becomes the router's explicit responsibility, with the full request in view. This part did not hold. The router had to make the same ambiguous call the agent did, with less context about the caller's actual goal — we relocated the ambiguity rather than removing it, and paid an inference hop for the privilege.
Mutations funnel through one gate. Every state change passes the same confirm step, enforced once instead of bolted onto every dangerous tool. This part backfired. A blanket policy taxes every harmless read to protect a handful of destructive calls, and the paraphrase that makes the gate legible is a lossy summary — which means a partial write reads exactly like a complete one. Per-tool gates on the irreversible verbs turned out to be both safer and cheaper.
The tradeoffs — paid honestly
A gateway is not a free lunch. Be clear-eyed about what it costs.
An extra LLM hop. Routing is itself an inference. You've added a model call between intent and execution, with its own latency, its own token cost, and its own failure modes. The router can misroute just as a direct agent can mispick — you've relocated the ambiguity, not abolished it. The bet is that a model whose only job is routing, with the full request in front of it, does that job better than an agent doing it as a distraction. That bet usually pays off, but it is a bet.
Latency. Two hops are slower than one. For interactive, read-heavy use this is often invisible against network and tool time. For tight latency budgets it matters, and you should measure rather than assume.
You now need a confirm flow. Routing to a mutating action means you need somewhere to stash the pending action, surface the paraphrase, and apply or drop it on the agent's response. That's real machinery — state for the pending action, a clean way to confirm or cancel — and it's mandatory, not optional, the moment a router can trigger writes on inferred intent.
Debuggability moves. With direct tools you read one call. With a gateway you trace intent → routing decision → invocation → result. Good logging at the routing layer makes this tractable; without it, you've added a black box.
The third option we should have tried first
The framing above has a hole in it, and we didn't see it until the data forced the issue: it treats the catalogue size as fixed. Both branches assume you have fifty tools and argue about how to present them. The cheapest fix is to not have fifty tools.
Merge the confusable families. Four tools that read different views of one object should be one tool with an include parameter. That removes the ambiguity at the source instead of hiring a router to resolve it. We collapsed our own ticket-reading cluster this way and the disambiguation problem simply stopped existing.
Serve a profile, not the whole list. The MCP tools/list call can return a curated core set by default and the full catalogue on request. That is most of the gateway's context saving, with no inference layer between intent and execution.
Gate the dangerous calls individually. Destructive verbs can require an explicit argument to fire — ours refuse to run without allow_writes: true, and the delete that agents call most often archives rather than destroys. Five careful tools beat a confirm step in front of forty.
Where we landed
Reach for direct, typed tools as the default — but keep the catalogue curated. That means merging overlapping tools rather than accumulating them, writing names and descriptions precise enough that a model doesn't need a paragraph to choose, and serving a core profile so the context cost tracks what agents use rather than what you've built.
Reach for a gateway only after you have curated and measured, and the tax is still unacceptable. The honest rule of thumb we'd offer now is the inverse of the one this post shipped with: if you find yourself writing per-tool documentation explaining which of several similar tools to use when, that's the signal to merge those tools — not to put a router in front of them.
AppHandoff's MCP server at https://api.apphandoff.com/api/mcp-bot now runs 24 listed, directly callable typed tools with no routing layer. What the switch cost and what the production numbers were is in the retrospective; the current surface is on the MCP server page.