AI Agent Memory: Four Types Every Architect Must Know

The model is a solved problem in one sense: you pick a frontier model, tune the system prompt, and you have coherent reasoning. Memory is different. There is no single memory API. Every agent system is a custom patchwork of context window management, retrieval pipelines, session stores, and tool registries. Each layer can fail independently, and the failures are often silent.

A customer support agent that forgets the conversation two messages ago, a coding assistant that cannot recall last week's architecture decision, a research agent that retrieves the wrong document because a chunk boundary cut a key sentence in half — these are all memory failures. They look like model failures to users, which is why they are costly to debug.

The foundation of solid AI agent memory management is a clear taxonomy. There are four distinct memory types. Each stores different information, operates at a different timescale, and breaks under different conditions.

It is the fastest memory an agent has. Zero retrieval latency, zero external dependencies. The model sees everything in context directly, without a lookup step.

The limit is the window size. Modern frontier models support 128k to 1 million tokens in principle, but retrieval quality degrades past 50 to 70 percent capacity on most current models — the model attends poorly to content near the center of a long context. Longer contexts also cost more: a 200k token context costs roughly 10x a 20k context at current API pricing.

Three production failure modes to know:

For most agents, in context working memory is the only type that runs in every call. The other three extend what it can know and recall beyond that window.

When the agent needs a fact, a document, or a past decision that would not fit in context, it encodes the query as a vector embedding, runs a similarity search against a vector index, and retrieves the nearest chunks. Those chunks are then injected into the in context window for that specific call.

Common vector store options in production: Pinecone for fully managed hosted retrieval at scale, Weaviate for teams that want to run their own instance with hybrid search, pgvector when you already run PostgreSQL and want to avoid a separate service, Chroma for local development and testing. Each has different tradeoffs around latency, cost, and the ability to combine semantic search with metadata filters.

Chunking strategy is the most underrated decision in the pipeline. Retrieval quality depends on whether chunks are coherent units of meaning or arbitrary byte slices. Recursive character splitting with 512 to 1024 token chunks and 20 percent overlap works well for unstructured prose; document aware splitting is better for structured content like specs and code.

Three production failure modes:

The challenge in agent systems is that retrieval happens mid run, inside a loop, often based on intermediate outputs — not just the user's original query. Designing the retrieval trigger is as important as the retrieval infrastructure itself.

Without this layer, every session starts cold. The agent has no knowledge of what you asked yesterday, what it proposed last week, or why it was corrected. For a customer support agent or a daily coding assistant, cold start is a serious usability failure.

The implementation pattern: after each session ends, distill the conversation into a structured summary and write it to a persistent store (PostgreSQL, Redis with TTL). When a new session begins, retrieve relevant summaries and inject them into the system prompt or early context. Distillation matters. A good session summary captures what the user was trying to do, what the agent did, what worked, and any expressed preferences or constraints. Typical compressed size: 200 to 400 tokens per session.

LangGraph's memory module supports session persistence via Postgres and Redis checkpointers. Some teams go further and index past session summaries in a vector store, letting the agent search its own history semantically rather than just by recency.

Think of it as the agent's encoded skills and constraints. A customer service agent's procedural memory includes: which tools it may call, the policy requiring escalation for refunds above $500, and last month's correction not to recommend a discontinued product.

In practice, procedural memory lives in several places. Tool schemas are the most structured form: each tool carries a JSON schema describing what it does and what arguments it accepts. System prompt policies are natural language behavioral rules. Correction logs are the trickiest part: if a user tells the agent “never do X again” and you want that correction to persist across sessions, you need to write it somewhere and inject it into future sessions.

Procedural memory is the layer that most distinguishes a production agent from a prototype. The prototype has hard coded behavior. The production system has a memory layer that lets behavior evolve — safely, with version control and review gates. In systems of multiple cooperating agents, procedural memory also governs how agents hand off context to each other; the post on multiagent design patterns covers that coordination layer in depth.

Episodic memory pays off only when the agent has repeat interactions with the same context over time. A document processing pipeline runs once per document and does not benefit. A personal coding assistant used daily benefits enormously.

External retrieval is worth adding whenever the knowledge base is larger than fits comfortably in context — typically anything over a few hundred pages, or any codebase with more than 50 files.

Start with in context management: nail window size, trimming, and structure. Add retrieval if the knowledge base demands it. Add episodic if sessions repeat. Procedural is wired in from the start and iterated continuously as behavior policies evolve.

For a broader view of how these memory layers compose with routing, orchestration, and safety controls, the guide on production agentic AI architecture covers the full system design.

If you are designing memory architecture for a production agent system, the agentic AI consulting service covers memory layer design, retrieval pipeline architecture, and the safety controls that keep agent behavior predictable as context accumulates.