An MCP registry is a catalog: it tells humans and agents which MCP servers exist, what version they are, and who owns them. An MCP gateway is a runtime proxy for tool traffic: authentication, authorization, rate limiting, routing, and audit logs for every tool call an agent makes. An AI gateway sits in front of model provider APIs instead — cost attribution, caching, key management, and provider failover — and is not MCP-specific at all.

Most teams do not need all three, and almost nobody needs them in the order vendors pitch them. Under roughly five MCP servers with one team, direct connections are fine and every layer here is premature. Past that, the trigger is not server count alone: a registry becomes necessary when people can't find things, an MCP gateway becomes necessary when security asks who can call what, and an AI gateway becomes necessary when finance asks where the model spend went — which usually happens first.

What does each layer actually do?

Four things get conflated in most comparison posts. Keeping them separate is the whole job.

MCP server — the individual endpoint that exposes a capability (query this database, file this ticket, read this repo) over the Model Context Protocol. This is the unit of work, and it's the layer below everything discussed here. If you're still deciding whether to build one at all, start with our guide on MCP servers for AI agents: what they cost and when you actually need one.

MCP registry — a searchable catalog of those servers, plus metadata: owner, version, health, security-scan status, intended audience. A registry is passive. It does not sit in the request path, it does not authorize anything, and it cannot stop a bad call. AWS's own writeup on governing AI assets at scale frames the problem a registry solves precisely: without one, "developers cannot find the right tool, agents cannot select the right peer, and users fall back to hardcoded URLs" (AWS Open Source Blog).

MCP gateway — an active proxy between agents and MCP servers. Agents connect to one endpoint instead of twenty. The gateway handles identity, tool-level access control ("this agent may call search_tickets but not refund_order"), rate limits, and an audit trail of which principal invoked which tool with what arguments. This is the layer security cares about.

AI gateway — a proxy in front of OpenAI, Anthropic, Bedrock, or your self-hosted models. It centralizes API keys, attributes spend to teams, caches repeated completions, enforces token budgets, and fails over between providers. Some products bundle MCP gateway features into an AI gateway; that's a packaging decision, not a category truth. The two solve unrelated failure modes.

At what scale do you need an MCP registry?

Roughly once you cross 10–15 MCP servers, or the moment a second team starts consuming servers a first team built. The team-boundary trigger fires earlier and matters more than the count.

These are engineering-judgment thresholds, not laws — calibrate them to your org. But the mechanism is consistent: with five servers and one team, the catalog lives in someone's head and a README. With two teams, the README goes stale within a sprint. Someone rebuilds a Jira MCP server that already exists. Someone points an agent at a v1 server deprecated a month ago. Nobody notices until an agent starts returning wrong answers in production.

The ceiling is high once you have one. AWS describes an Expedia Group deployment running "hundreds of MCP servers, tools, and skills alongside tens of agents in a searchable, graded, and centralized catalog" (AWS Open Source Blog) — the shape of the problem at the top end, and why registry-first ordering makes sense for platform teams whose main pain is sprawl rather than security.

A registry is also the cheapest of the three to start: a git-backed manifest with CI validation and a small search UI covers the first 80% of the value, and it's the version we most often recommend before anyone signs a contract.

At what scale do you need an MCP gateway?

Not at a server count — at a credential event. The moment two or more agents are running in production against MCP servers that hold real credentials to real systems, you need a gateway, even if you only have four servers.

The forcing question is always the same and it usually comes from security or audit: who can call what, and can you prove it after the fact? Without a gateway, the answer lives in per-server config scattered across repos, and the audit trail — if one exists — is per-server logs with no shared principal identity. You cannot answer "show me every write operation agent X performed last Tuesday" from that.

The governance gap here is measurable. Deloitte's State of AI in the Enterprise survey of 3,235 business and IT leaders across 24 countries found that only 21% of companies planning to deploy agentic AI within two years report having a mature model for agent governance (Deloitte). That gap is exactly the space an MCP gateway occupies.

Secondary triggers, any one of which is sufficient: agents with write access to customer-facing systems; regulated data flowing through tool calls; more than one team operating agents against shared servers; or a requirement to revoke an agent's access to one tool without redeploying anything.

At what scale do you need an AI gateway?

Usually earlier than either MCP layer — often at the second team, or the first month the model bill is large enough that someone asks for a breakdown by product.

An AI gateway is a cost, reliability, and key-management concern, and those arrive before governance does. The concrete triggers: you can't attribute spend per team or per feature; API keys are pasted into more than a couple of services; you've had a provider outage or rate-limit event with no failover path; or two teams are independently paying for near-identical completions with no shared cache.

Note the independence: a team with a single MCP server and heavy model usage needs an AI gateway and neither MCP layer. A team with thirty MCP servers behind one internal model deployment may need a registry and a gateway and no AI gateway at all. Any comparison presenting these as tiers of the same product is selling you something.

What breaks if you skip each layer?

• No registry, 15+ servers, multiple teams. Duplicate servers, agents wired to deprecated versions via hardcoded URLs, a security fix shipped to one of three copies. Onboarding an engineer takes a week of Slack archaeology.
• No MCP gateway, agents in production. Over-broad tool access nobody reviewed — the classic being an agent granted a full database server when it needed three read queries. No unified audit log, so incident response starts with reconstructing what the agent did from application logs. Revoking access means a coordinated redeploy.
• No AI gateway, multiple teams. Model spend that can't be attributed and therefore can't be controlled. No shared cache, so identical calls get paid for repeatedly. A provider outage becomes a product outage.
• All three, at ten engineers and four servers. Three systems to operate, extra latency on every tool call, and a platform team maintaining infrastructure for a problem that hasn't arrived — a real, under-discussed failure mode nobody selling a gateway writes about.

Should you build or buy each layer?

Buy when the operational overhead of running it yourself exceeds the vendor cost. That's a when, not an if — and it lands differently per layer.

Registry: build first, almost always. A schema, a git repo, CI validation, and a search endpoint is a small project, and requirements are org-specific enough that vendor defaults rarely fit your ownership model.

MCP gateway: the least attractive to build from scratch — you're implementing authorization, audit, and protocol-level proxying, security-critical code with real ongoing maintenance. Credible open-source options are the sensible middle path: self-hosting gets you the control plane without writing the authorization engine.

AI gateway: the most mature category, with well-established open-source and managed options. Building your own is defensible only with unusual routing or data-residency needs.

One rule across all three: if a layer would be operated by less than a full-time engineer's attention, buy or self-host open-source. Half-maintained infrastructure in the request path is worse than none.

How do you work out where you actually are on this curve?

The assessment is short and mostly non-technical: inventory every MCP server and model integration actually in use (always higher than expected), map which agents and human principals can reach each one today, then identify which of the three trigger questions your organization is currently failing — discovery, authorization, or cost attribution.

That produces an ordering rather than a shopping list. Most teams need exactly one layer now, a second within two quarters, and the third not at all — a sequenced plan with build-vs-buy called per layer, so the infrastructure line item you take to your CTO covers one thing you can justify rather than three you can't.

If you're being pitched a platform that claims to solve all three, the useful counter-question is: which of my three trigger questions is failing today, and what does this product do about the other two that I couldn't do with a manifest file?

FAQ

Is an MCP gateway the same as an API gateway?

No, though they share mechanics. An API gateway routes and secures HTTP endpoints; an MCP gateway understands MCP semantics — tools, resources, prompts, and sessions — so it can authorize at the individual tool level rather than the route level. Some API gateways have added MCP support, which is a reasonable path if you already operate one at scale.

Can one product be both an MCP gateway and an AI gateway?

Yes, several are, and that's fine as long as you evaluate each capability separately. The risk is buying a strong AI gateway with a thin MCP feature set because the marketing presents them as one thing. Score model-side features and tool-side features on separate lines.

Do I need an MCP registry if I only have a handful of servers?

Probably not as a system. Under about five servers and one team, a maintained README plus a config file gives you the same outcome with none of the operational cost. Adopt a real registry when a second team depends on servers they didn't build.

Which layer should I add first if I need more than one?

Add whichever answers the question your organization is currently failing to answer. If security is asking who can call what, gateway first. If engineers can't find or trust the servers, registry first. If nobody can attribute model spend, AI gateway first — and that one is most often the earliest trigger.

Are open-source options viable for production use?

For registries and AI gateways, yes, and they're often the right default. For MCP gateways, evaluate the authorization model and audit-log guarantees specifically, since those are the features you're buying and maturity varies widely across projects.

How much latency does an MCP gateway add?

Expect a single additional network hop, typically low tens of milliseconds within the same region, which is negligible against model inference time. The cost worth measuring is not latency but the blast radius of the gateway itself: it is now a single point of failure in front of every tool call, so plan its availability accordingly.