Rethinking AI Agent Architecture with MCP

Why you should consider using Anthropic's Model Context Protocol (MCP); an open source standard to connect large language models (LLMs) to external sources and tools.

If you haven’t already read Anthropic’s announcement, here’s how they describe their open source protocol for connecting LLMs to external tools and sources:

Think of MCP like a USB-C port for AI applications. Just as USB-C provides a standardized way to connect your devices to various peripherals and accessories, MCP provides a standardized way to connect AI models to different data sources and tools. - Source

Ok, huh? USB-C ports? Let’s give MCP the credit it deserves and walk through the history that got us here, what MCP is, how it tries to solve for tomorrow, and what are some benefits if you choose to adopt this protocol in its early days.

The History

GenAI

This was the ChatGPT revolution that made “AI” everyone’s favorite word. Ask an LLM anything and it’ll respond based on the information the model was trained on. Cool.

RAG (Retrieval-Augmented Generation)

LLMs could understand and produce code, so what was stopping us from asking LLMs to generate database queries, curl calls, scripts, etc.?

In the RAG era, engineers called LLMs directly in code, injecting custom context with the user’s inquiry to get some structured response that allowed them to programmatically call a backend API, database, etc.. Here’s a quick example:

Given the SQL table definition below, generate a SQL query that can answer the following question: what movies are rated 8 or higher?

Please return nothing other than the SQL query.

CREATE TABLE movies (
id SERIAL PRIMARY KEY,
name VARCHAR(255) NOT NULL,
rating DECIMAL(2,1) CHECK (rating >= 0 AND rating <= 10),
storyline TEXT
);

This worked great, but scaled poorly as you added more tools or actions you wanted the LLM to use when responding to a prompt. Your engineers were also on the hook for formatting inputs/outputs and juggling all the possible prompts and their relevant contexts.

Agents

The shortcomings of simple RAG implementations were quickly addressed with the advent of AI agents. Open agent frameworks (e.g. PydanticAI, Agno, crewAI, AutoGen, LangGraph, LibreChat) and managed services (e.g. AWS Bedrock Agents) handled the dirty work of integrating with LLM providers, managing configuration for tools/integrations, and injecting the appropriate context to facilitate a RAG-like experience.

Agents are really awesome! But they came with a big caveat:

You are pigeonholed into the integration patterns of the framework or SaaS solution you pick.

That’s not always a bad thing, but given how quickly the AI landscape is evolving, it’s probably not the best idea to get trapped by vendor lock-in or set your team up for major tech debt every 2-3 years.

Your evolution of agent frameworks could look like this:
LangGraph (Y1) → Bedrock Agents (Y2) → PydanticAI (Y4) → Something Else?!

Now imagine re-implementing, configuring, testing, and scaling your 100+ agent integrations every few years when your framework changes! 🤦‍♂️

Introducing MCP

Just like any other protocol (e.g. HTTP, SSH, FTP, etc.), MCP attempts to set an open standard protocol for how AI clients/agents talk to systems that have useful information like databases, APIs, file/object storage, etc..

The MCP standard defines two parties:

• MCP Client: Any user-facing application that is integrated with an LLM

• MCP Server: A sidecar to any service or source system that you would like to make available to the client

Using the MCP server library, you can define a sidecar that lives alongside your source system and provides just enough context about the source system and what tools, capabilities, and actions you are making available to MCP clients.

Using the MCP client library, you can integrate your client facing AI application with 1 or more MCP servers (as simple as specifying the MCP server’s URL). Assuming your agent framework already has a built-in MCP client (many already do), your framework’s MCP client will auto-discover the tools and actions exposed by your MCP server(s) and make them readily available to the agent/LLM when responding to user prompts.

That was a lot in three paragraphs, but I hope it gave you the gist. We’ll cover setting up an MCP client and server in future posts, so stay subscribed to the newsletter!

MCP Community

MCP has an active community that has built a vast array of MCP servers available for you to fork or use directly. There’s support for popular services like S3, MongoDB, PostgreSQL, GraphQL, etc.. Take a look at this awesome, community aggregated list of open source MCP server implementations: https://github.com/punkpeye/awesome-mcp-servers.

MCP Connectivity

Early Days (stdio)

MCP’s original intent was to integrate Anthropic’s consumer-grade Claude desktop app with local services (e.g. operating system, web browser, file search, etc.). It was initially released with standard I/O being the underlying protocol between the client and server(s).

This heavily limited its usefulness for enterprise use cases where servers are typically remote.

2025 (SSE)

MCP now supports client to server communication over SSE on HTTP(S). This means you can deploy your MCP server like any other API service (e.g. in your k8s cluster) and reach it from any MCP client anywhere (web, desktop, mobile, IoT).

This also means you can add your own security around the MCP server:

• HTTPS

• Firewall (WAF)

• Auth (OAuth2, Basic, etc.)

• Rate Limiting

• Logging

Backwards Compatibility (stdio → SSE)

Projects like supergateway make it easy to take an existing stdio MCP server and run it as an MCP server over SSE:

npx -y supergateway \
    --stdio "npx -y mongodb-lens mongodb://your-connection-string" \
    --port 8000 --baseUrl http://localhost:8000 \
    --ssePath /sse --messagePath /message

The Benefits

Let’s quickly recap the benefits of introducing MCP to your architecture:

• Create reusable tools and actions that can be instantly integrated across multiple AI agents and frameworks.

• Scale your agents and tools individually.

• Framework agnostic - Use whatever AI framework you want today and swap it out with minimal effort in the future.

• No vendor lock-in - Be agile and adaptable to new AI tech. Use the best underlying services where they are available (AWS, GCP, startups, homegrown) as long as an MCP server can reach it (VPC or public internet).

• Threat mitigation - If your agent or AI client is under attack, only take down your MCP server. Keep your source systems operational and resilient.

• Use your preferred, existing security model with firewalls, auth, rate limiting, etc..

• Audit and log at the MCP server level to prevent log pollution and separate agent-initiated logs from your application and service logs.

• Foster a micro-services approach to building enterprise-grade AI agents that can otherwise seem very monolithic and messy.