Memory for AI agents: a practical architecture guide

AI memory is an application capability, not a larger context window. The application chooses information worth retaining, associates it with a durable identity, and retrieves relevant material when a later request needs it. The model still receives a finite prompt. Memory changes how that prompt is assembled by making useful information available across conversations, jobs, devices, or model providers instead of leaving it trapped in a single transcript.

A memory record can be a concise preference, a decision, an observed fact, a summary of earlier work, or a passage from a source document. It should not silently replace an authoritative account record, policy repository, or transactional database. Those systems remain responsible for current truth. Memory is best used as a retrieval layer that helps an agent find context and evidence, while the application decides when to re-check a system of record before answering or acting.

A basic loop has four steps. First, the application identifies a stable scope such as a user, customer, project, or agent session. Second, it sends selected text or source material to the memory layer. Third, the layer indexes that material so it can be found by meaning rather than only by an exact keyword. Finally, before the next model call, the application recalls a small set of relevant results and places them in the prompt with enough context to interpret them.

The important design choice is selectivity. Saving every token creates noise, cost, and conflicting history. Saving nothing forces the user to repeat themselves. A practical policy records information that is likely to matter later, such as a stable preference or an explicit project decision, and keeps transient small talk out. Recall should also be bounded: retrieve a small number of useful items, rank them, and let the agent say when the stored context is insufficient.

Memory isolation starts before retrieval. Aether uses the API credential to identify a tenant, supports an optional partition inside that tenant, and requires an entity ID for Memory operations. The SDK attaches the entity ID from the Memory handle to its requests. That structure helps application code keep one customer or agent scope from being mixed with another, but it does not decide who is allowed to select a particular identifier.

Authorization remains an application responsibility. A server should derive the permitted tenant, partition, and entity from the authenticated caller or trusted workflow; it should not accept an arbitrary scope from an untrusted browser and assume the memory service will understand the business relationship. Use stable opaque identifiers, document the mapping to your own accounts or workspaces, and test that one principal cannot request another principal’s memory even when it can guess an identifier.

A context window is the information a model can inspect during one inference. A memory layer helps choose what to put there. Retrieval-augmented generation, or RAG, is the broader pattern of finding relevant external material and adding it to a prompt. Vector search is one retrieval technique: it compares numerical representations so related meanings can match even when the words differ. These concepts overlap, but they are not interchangeable.

Agent memory usually adds an identity and lifecycle around retrieval. The application needs to know whose information is being stored, when it was learned, whether it is still valid, and how it should be removed. Document RAG usually begins with a corpus such as manuals, policies, or reports and retrieves supporting passages. Aether exposes a Memory facade for entity-scoped remember and recall operations and lower-level document and search APIs when an application needs direct control over its corpus.

Good candidates for memory are compact, attributable, and likely to change a future answer. Examples include a user’s communication preference, the decision behind a project convention, the summary of a resolved support issue, or a source passage that should be cited later. Keep the original source or a durable reference when accuracy matters. A bare sentence without provenance may be easy to retrieve but hard to trust once circumstances change.

Treat inferred information more cautiously than explicit information. If a user directly states a preference, the application can label it as user-provided. If an agent infers a preference from behavior, preserve that distinction and consider confirmation before relying on it. Avoid retaining secrets merely because they appeared in a conversation. Your memory policy should define allowed data, retention, correction, and deletion alongside the code that calls remember.

A successful API response only proves that a record was accepted. It does not prove that future recall will select the right item or that the model will use it correctly. Build a small evaluation set from realistic tasks: facts that should be recalled, distracting items that should not win, conflicting updates, empty-history cases, and attempts to cross a scope boundary. Measure retrieval quality separately from the final model response so failures remain diagnosable.

Operational checks matter too. Track failed writes, empty recalls, unusually broad result sets, and latency at the point where memory is added to a prompt. Define how corrections supersede stale facts and how deletion propagates through stored source material and derived indexes. Keep a no-memory fallback so an upstream problem does not force the agent to invent context. Durable memory should make the system more dependable, not give it a new reason to answer with false confidence.

Start with one outcome that is easy to inspect. A support assistant might recall a customer’s preferred response format. A coding agent might retrieve an accepted repository convention. Give the workflow one stable entity ID, retain only the minimum useful fact, and show the retrieved item in logs or a review surface before trying to automate a wider memory policy.

Aether provides first-party Python, TypeScript, Go, and .NET SDKs. Each SDK exposes the same entity-scoped Memory idea, while the document client remains available for source-grounded retrieval. Follow a language quickstart, verify a remembered fact can be recalled in a fresh session, then add authorization and negative-scope tests before connecting memory to a production agent.