We're hiring! Come build with us
Zep
AI Agents Guide

How to Give an AI Agent Long-Term Memory

Give an AI agent long-term memory with a temporal context graph that persists facts across sessions and serves relevant context per turn. Approaches compared.

Chat
RRobbie2024-09-07
I only wear Adidas shoes. I love them!
Business data
soleworks.com/returns/SO-48219
SoleworksReturn · Adidas Ultraboost 22

Reason for return

Product fell apart
These Adidas fell apartafter three weeks. I’ll be buying Nike from now on.
FactsExtracted · 3
  • Robbie strongly favors Adidas shoes.
  • Robbie’s Adidas Ultraboost 22 fell apart.
  • Robbie will buy Nike next.
EntitiesRelationshipsTimeline

Agent memory is what an AI agent knows across time about the user, the business, and the work. Long-term memory is the part that persists between sessions. You give an agent long-term memory by adding a memory layer that holds what the agent learns and serves the relevant slice back into the prompt at run time. Appending chat history to the context window does not scale: it grows without bound and mixes stale facts with current ones. A temporal context graph instead records facts with provenance and a validity window, and retrieves only what's relevant for the current turn.

Key takeaways

  • Give an agent long-term memory with a memory layer that persists facts across sessions and serves the relevant slice per turn, rather than stuffing chat history into the context window.
  • Use a temporal context graph: every fact carries provenance and a validity window, so the agent can retrieve what's true now and what was true then. Add it in a few lines of code, with any framework.
  • At enterprise scale, long-term memory needs governance: attribute-based access control, retention, and audit, with low-latency retrieval. Zep implements this as a Context Lake and reports 94.7% on the LoCoMo benchmark at sub-200ms p95 (results).

The approaches, compared

ApproachHow it worksBreaks down when
Stuff full chat history into the promptAppend every turnContext window fills; cost and noise rise; stale facts contradict current ones
Summarize historyPeriodically compress the transcriptLossy; loses specifics; no provenance or “as of” time
Vector store / RAG over historyEmbed turns, retrieve similar ones“Similar” is not “needed”; no relationships; no temporal reasoning
Temporal context graphExtract facts and relationships with validity and provenance; retrieve relevant current contextFits the job: handles change, relationships, and provenance

The three types of long-term memory

Long-term memory for an agent comes in three kinds, borrowed from how cognitive scientists describe human memory. Semantic memory holds facts and preferences independent of when they were learned, like a user coding in Go or an account on the Pro plan. Episodic memory holds time-stamped events: a specific message, a support ticket, a plan change on a given date. Procedural memory holds the routines an agent reuses, such as the steps it follows or the output format a user expects.

Most production agents need all three. Zep builds the first two from one temporal context graph. Episodes are the raw, lossless record of every input, the episodic layer. Entities and facts, each carrying a validity window, form the semantic layer. Observations are derived patterns across many episodes: recurring behaviors and decisions that no single fact holds. Procedural routines stay in the agent's own code and prompts, and Zep serves the facts and patterns those routines depend on.

How long-term memory works inside an agent

Inside a running agent, long-term memory works as a loop: read before reasoning, write after acting.

  1. Receive the input (a user message, a trigger, or an upstream agent).
  2. Read memory: retrieve the relevant slice from the long-term store and assemble the prompt.
  3. Reason and act: call the model and run tools.
  4. Write memory: extract new facts from what happened and update the store.

The hard part is step 2. Given everything that could go into the prompt, the system has to choose what a single step needs. The steps below put this into practice.

How to add long-term memory, step by step

1. Choose a memory layer over a bigger window

A larger context window lets you pass more text. It is not memory: you still have to decide what to pass. A memory layer builds persistent structure from the agent's inputs and selects the relevant context each turn.

2. Ingest every source the agent touches

Long-term memory should unify more than chat: messages, business data (JSON), and documents. Feeding all sources into one graph is what lets the agent know the user and the business, not just the conversation.

3. Let the system build a temporal context graph

Rather than holding raw text, the system extracts entities, relationships, and facts into a bi-temporal graph. Each fact records where it came from (provenance) and when it was valid. When a fact changes, the old one is invalidated rather than overwritten, so the agent can answer “what's true now?” and “what was true then?”

4. Retrieve relevant context at run time, not the whole history

On each turn, fetch the token-efficient, relevant slice of memory and add it to the prompt. This keeps the context window clean and latency low, and it is the step that reduces hallucination.

8 candidates · ranked for task
ObsJane upgrades within 2 weeks of each launch.
FactJoined Aug 2024.
FactCurrently on Pro v4.
FactAccount billing monthly.
SumRecent chats: power-user features.
SumPast tickets: rate limits.
ObsTickets pair with plan changes.
FactLast login 12h ago.
Context block1,847 / 2,000
ObsJane upgrades within 2 weeks of each launch.
FactCurrently on Pro v4.
SumRecent chats: power-user features.
ObsTickets pair with plan changes.

5. Add memory in a few lines of code

With Zep, this is three lines and works with any agent framework, or none:

# Add the turn to memory and get assembled context back in one call
response = client.thread.add_messages(
    thread_id=thread_id,
    messages=[Message(name="Jane", role="user", content="I'd like to upgrade my plan...")],
    return_context=True,
)

# Add business data to the same user's graph
client.graph.add(user_id=user_id, type="json",
                 data=json.dumps({"event": "plan_upgrade", "to": "pro", "mrr": 49}))

# Retrieve relevant context for the next turn
user_context = client.thread.get_user_context(thread_id=thread_id)

6. Govern and scale it

In production you need more than storage: access control over what each agent can see, retention policies, audit, and millisecond retrieval across many users. That is the difference between a memory feature and memory infrastructure, a Context Lake.

Tracking what's true over time

The hardest part of long-term memory is not holding facts. It is knowing which ones still hold. A vector store retrieves text by similarity and has no notion of time, so a preference the user changed last week sits next to the current one. A bi-temporal graph solves this by recording when each fact became true and when it stopped being true. When a fact changes, the old version is marked invalid rather than overwritten, so the agent can ask what is true now or what was true on a given date and get the right answer to either. Invalid facts leave retrieval on their own, so the agent is never fed stale context. Zep implements this model on a temporal context graph.

What belongs in long-term memory

Long-term memory is most valuable when it is fed more than chat. Useful sources to ingest into the same user graph:

  • Conversation — every user and assistant message (the running relationship).
  • Business data— transactions, plan changes, support tickets, app events, CRM records. This is what lets the agent know the user's situation, not just what they typed.
  • Documents — emails, transcripts, and files tied to the user.

You can send any of it as messages or via graph.add (JSON, text, or message). The system extracts entities and facts from all of it into one temporal graph, so retrieval can draw on the whole picture.

Retrieving the right slice (including patterns)

At run time, the agent pulls the assembled context, and when useful the derived Observations (cross-session patterns), via a dedicated search scope:

# Assembled, token-efficient context for the current turn
context = client.thread.get_user_context(thread_id=thread_id).context

# Or query derived patterns directly
patterns = client.graph.search(
    user_id=user_id,
    query="What does this user do before they upgrade?",
    scope="observations",
    limit=5,
)

The agent receives the relevant facts, entities, and patterns, never the entire history.

Agent memory benchmarks: LoCoMo and LongMemEval

LoCoMo and LongMemEval are the two public benchmarks for long-running agent memory. They measure whether a system retrieves the right facts across many sessions and over time (LoCoMo paper; LongMemEval paper).

On LoCoMo, Zep answers 94.7% of questions correctly (1,459 of 1,540) at 155ms p95 retrieval, with a median 5,760-token context. On LongMemEval it reaches 90.2%. For agents that prefer one call over multi-scope retrieval, Zep's auto search returns 86.5% on LoCoMo at 173ms p95 with a 2,680-token context, roughly half the tokens. Full methodology and per-category results are on the research page.

What long-term memory gives the agent

An agent that stays consistent when facts change and grounds its answers in what's known about the user and the business. It remembers decisions across sessions instead of re-deriving them every turn. Zep is the Context Lake for AI agents: it manages, governs, and serves agent memory on temporal context graphs, built on the open-source Graphiti and running on Konig, Zep's proprietary graph database service, with sub-200ms p95 retrieval.


Related: What is agent memory? · Agent memory vs RAG · What is a temporal knowledge graph? · Quickstart (docs) · Benchmark results · AI agent memory guides

Frequently asked questions

What's the simplest way to give an agent long-term memory?

Add a memory layer that builds a context graph from the agent's inputs and returns relevant context per turn. With Zep it's three lines of code, framework-agnostic.

What are the types of long-term memory for an AI agent?

Three: semantic (facts and preferences), episodic (time-stamped events), and procedural (reusable routines). A temporal context graph builds the first two from the agent's inputs, with facts and entities carrying validity windows for semantic memory and episodes for episodic memory, then derives cross-session Observations on top. Procedural routines live in the agent's own code.

Why not just use a bigger context window?

More tokens is not memory. You still have to choose what to include, and dumping full history adds cost and contradictions. Memory selects the relevant, current facts.

Can I use long-term memory with my existing agent framework?

Yes. A good memory layer is framework-agnostic and works with LangGraph, custom agents, or none.

How is this different from RAG?

RAG retrieves static documents by similarity. Long-term agent memory tracks evolving, provenance-stamped facts about the user and the business over time. Use both: RAG for documents, memory for state.

What should I keep in long-term memory?

Conversation, business data (transactions, tickets, events), and documents, all ingested into one per-user temporal graph so the agent knows the user's full situation, not just the chat.

Where does long-term memory live?

Outside the context window, in a durable per-user store (a temporal context graph). At enterprise scale that store is a Context Lake, governed and served in milliseconds.

How do I surface patterns across sessions, not just facts?

Observations capture the patterns surfaced across the graph: recurring behaviors, decisions, and preferences that no single fact holds. The agent retrieves them via the Observations search scope or a context template.