Building persistent memory for AI agents with Qwen and MCP

Most AI assistants reset every session. That is fine for one-off questions. It breaks down when you want a coworker that remembers how you work, what you decided last week, and what still needs follow-up.

At John CEO, memory is not a nice-to-have. Each customer gets a private workspace where context should survive restarts, not evaporate when a chat window closes. That product vision is what drew us to the Qwen Cloud Hackathon (Track 1: MemoryAgent): a chance to prototype how semantic memory might work when exposed as standard agent tools.

What follows is a standalone learning experiment, not a product announcement. The code lives in a separate MIT-licensed repo (qwen-memory-mcp). John CEO does not use it today. In production, memory stays on each client's individually isolated workspace, not on a shared external service. We built this to learn, stress-test ideas, and share what we found.

What we set out to learn

We wanted durable memory that is:

• Durable - survives restarts and redeploys
• Semantic - finds related facts, not just keyword matches
• Scoped - per user, not mixed with anyone else's data
• Simple to integrate - standard MCP tools, not a custom SDK

The hackathon required real Alibaba Cloud infrastructure and Qwen intelligence. Our answer was a small MCP server: store memories in a managed SQL database, embed them with Qwen, and expose four HTTP-callable tools any agent client can use.

Architecture

An agent client calls a public HTTPS endpoint on Alibaba Function Compute. The memory service writes rows to a managed SQL database and calls Qwen Cloud (DashScope) for embeddings and ranking. The design is intentionally small: standard MCP over HTTP, with a SQL store behind it and Qwen handling semantic work.

The experiment runs in ap-southeast-1 (Singapore), Alibaba Cloud's international region. We kept the Qwen API endpoint, compute, and database in the same region, outside mainland China. For a European team building for global users, that regional choice was deliberate: it matches the international DashScope endpoint and aligns with how we think about cross-border data handling. This is not a compliance certification - it is a practical fit for an experiment aimed at international use.

Four MCP tools

Four tools beat a sprawling API. Agents need write, search, pack-for-context, and forget - not twenty CRUD variants with overlapping semantics.

Centralized memory vs workspace memory

We compared this centralized MCP experiment with the workspace-local memory we ship for John CEO customers. Both aim at the same goal - a coworker that remembers - but the shape of the system changes the tradeoffs.

1. Isolation model. A shared memory service with per-user ids is fast to demo and easy to wire through MCP. Keeping memory on each client's individually isolated workspace keeps data boundaries tighter.

2. Where embeddings run. Cloud API embeddings simplify setup but add latency, cost, and cross-border data questions. On-workspace retrieval avoids shipping raw text to a third party.

3. Audience-aware recall. Real coworkers operate in channels and threads, not just a single user id. Memory that only namespaces by user is simpler but misses context that matters in team settings.

4. Ranking travels well. Combining similarity, salience, recency, and reinforcement improved recall in both shapes. The algorithm mattered more than the storage brand.

5. Maintenance is not optional. Forget and decay passes prevent memory from growing forever. Both centralized and local designs need an explicit lifecycle.

6. Tool surface. Four explicit MCP tools made agent integration predictable. Implicit file-based memory is flexible but harder for agents to use reliably.

What surprised us

1. Cold starts are real. The first MCP call after idle on serverless compute can take 10-20 seconds. That latency is part of the UX story, not a footnote.

2. MCP over HTTP is not plain JSON. Responses may arrive as Server-Sent Events. Our first smoke scripts assumed curl | jq would suffice. They did not. We ended up with a small Node parser so verification scripts could handle both shapes.

3. Semantic search changes the product feel. Keyword search fails on "how do I like client drafts?" when the stored memory says "keep external emails under three short paragraphs." Embeddings bridge that gap.

4. Token budgets matter as much as retrieval. memory_recall_context was not an afterthought. Real agents hit context limits fast. Packing the best memories into a budget beats dumping the top ten search hits into the prompt.

5. Small surface area wins. A tight tool list helps both agent integration and human operators. Four tools stay in working memory.

How John could use this (hypothetically)

Internally we sketched a wiring pattern similar to how connections work today: at workspace boot, hand the agent a scoped MCP URL and bearer token. The Qwen API key and SQL credentials would stay on Alibaba Cloud only - never on the customer's private workspace. Each user's memories would namespace by customer id on the shared memory server.

That blueprint is documented but not implemented and not on the roadmap. Production John memory stays on each client's individually isolated workspace. This experiment informed what we want algorithmically (ranking, token packing, scoped recall) without committing to Qwen or MCP in the live product.

Open source

The project is MIT-licensed on GitHub: qwen-memory-mcp

Clone the repo and follow the README for local runs, integration tests, and deployment notes.

Verify it yourself

git clone https://github.com/John-CEO-HQ/qwen-memory-mcp
cd qwen-memory-mcp
npm install
cp .env.integration.example .env.integration
# Configure env from .env.integration.example
npm run verify:deployed

You should see All checks passed. at the end. That run exercises health, tool listing, write, search, recall, and forget against the endpoint you configure in the repo docs.