// learn

Agent memory without a vector database

Somewhere along the way, "AI memory" came to mean embeddings and a vector database. For a large share of what agents actually need to remember, that is more machinery than the job requires. Here is how to tell which kind of memory you have, and why most agents can skip the vector database entirely.

Where the vector database assumption comes from

Vector databases earned their place through retrieval-augmented generation, where the job is to search a large body of freeform text and pull out the passages most relevant to a question. That genuinely needs semantic search: you cannot look those passages up by name because you do not know in advance what you are looking for, only what it should be about.

Because that pattern arrived first and loudly, "give my agent memory" quietly became "set up embeddings and a vector store." But remembering a fact about an agent and searching a corpus by meaning are two different jobs. Conflating them is how people end up running a vector database to store the fact that a user prefers dark mode.

The two kinds of memory

Almost every memory an agent holds falls into one of two buckets, and the bucket decides how much machinery you need.

Structured memory: you know the key

These are named values you store and retrieve by an exact key. You know what you saved and you know how you will ask for it later. The retrieval is deterministic: the same key returns the same value, every time, with no guessing involved.

• A user's language, tone, timezone, or settings, under a key like user_language.
• The current step of a multi-part task, so the agent can resume instead of restarting.
• Identifiers and results, such as last_invoice_id or the output of an earlier step.
• Flags and decisions the agent made and must not contradict later.
• Facts several cooperating agents share through a common namespace.

None of this needs similarity search, because you are never asking "what is roughly like this?" You are asking "what did I store under this exact name?" That is a lookup, not a search.

Semantic memory: you only know the meaning

This is memory you search by meaning because you cannot name what you want in advance. If an agent needs to find "the passages most relevant to this question" across a large body of text it has never indexed by hand, it needs semantic search, which is where embeddings and a vector database come in.

• Searching thousands of documents or long transcripts for whatever is relevant to a new query.
• Recalling loosely related past conversation without knowing which turn holds the answer.
• Finding items by resemblance rather than by an exact identifier.

This is more powerful and more complex, and plenty of agents never actually reach for it.

The test: can you name what you are storing?

You do not need to memorise a taxonomy. One question sorts almost every case.

"Remember this user's plan tier" passes the test: you will ask for it by name. "Find whatever the user once said that is relevant to this new question" fails it: you can only describe the target, not name it. The first is a lookup. The second is a search. Most agent memory is the first.

What a vector database actually costs you

A vector database is not bad. It is simply not free, and the cost is easy to miss until you are running one you did not need.

• An embedding step on every write and every read. Each value has to be converted to a vector by a model before it can be stored or matched, which adds a model call, latency, and cost to operations that could have been a single lookup.
• Infrastructure to run and scale. The store itself has to be provisioned, kept available, and maintained, or paid for as another managed service.
• Approximate results. Similarity search returns what is closest, not what is correct. For remembering a user's exact setting, "closest" is the wrong tool; you want the precise value you saved.
• More moving parts. An embedding model, a vector index, and the glue between them are three more things that can drift, fail, or surprise you, for memory that a named lookup would have handled outright.

When your agent genuinely needs meaning-based retrieval, that cost buys something real. When it needs to remember a value it can name, you are paying for machinery you will never use.

What you use instead

For structured memory, the whole thing is two operations against plain storage: write a value under a key, and read it back later. Scope the key to an agent or a user so tenants stay separate, add an expiry when a fact is only temporary, and share a namespace when several agents need the same value. No embeddings, no index, no database to stand up.

Over an HTTP API, each of those is a single request. Store user_language = French for a given agent now; read the user_language key back in a completely new session tomorrow and the agent knows to answer in French. The retrieval is exact and predictable, which is usually what you want from state. If you want the build-story version of this argument rather than the conceptual one, we wrote up building a memory API without a vector database separately.

When you genuinely do need one

Skipping the vector database is a default, not a rule. If your agent has to search unstructured text by meaning, that is exactly what semantic search is for, and you should reach for it. Retrieval over documents, knowledge bases, or long freeform histories is real semantic memory, and a purpose-built tool will serve you better than forcing a key-value model onto it. Our comparison of memory providers lays out where each approach fits.

Many agents end up wanting both: structured memory for the large, everyday share of state, and a semantic layer for the specific slice that is actually about meaning. The useful move is to keep them separate, so you add the vector database only for the part that needs it, and only once you actually do, rather than paying for it across everything from the start.