I'm exploring an application that gives AI conversations a persistent memory layer. Imagine working with an AI on a software project for an hour: you establish an architecture, make design decisions, fix bugs, and leave a few tasks unfinished. When you return shortly afterward, the AI may forget an earlier decision and suggest something that conflicts with the existing work.
Instead of relying only on the conversation history or a large context window, the system would maintain structured information such as the current objective, user requirements, important decisions, project state, completed and pending work, key facts, and dependencies. When the AI needs context, it would retrieve the most relevant memories and include them in the prompt.
Could this work as a general-purpose memory architecture for AI agents, allowing them to maintain reliable continuity across long-running tasks without resending the entire conversation every time? I'm especially interested in experiences with LLM agents, retrieval-augmented generation, vector databases, knowledge graphs, or other AI memory systems.
3 Answers
There are already several systems exploring this idea, including products described as AI memory layers and personal knowledge systems. Looking at existing approaches such as Second Brain-style tools and wiki-based solutions can reveal useful patterns. The difficult part is not simply saving information; it is deciding what deserves to be remembered, detecting when an old decision is invalid, and allowing the user to inspect or correct the stored memory.
A practical starting point is to treat the memory as project documentation: architecture decision records, task notes, requirements, and development logs. These artifacts are explicit, reviewable, and easier to correct than memories generated automatically by a model. An agent can be instructed to update them as the project changes, then retrieve the relevant sections later.
That makes sense, and I agree that ADRs and logs solve part of the problem. I'm mainly interested in automating the maintenance layer so the system can extract decisions, task state, dependencies, and other durable context from conversations instead of relying entirely on a developer or agent to record everything manually.
This is definitely feasible, but it helps to separate different kinds of memory rather than putting everything into one vector store. Durable facts and decisions can live in structured records, task progress can use a state machine or database, and semantic search can handle less structured notes. Retrieval should also be selective and grounded in the current task, otherwise the model may receive stale or irrelevant memories.
Exactly. A memory system needs confidence, timestamps, provenance, and some way to mark information as superseded. Otherwise it can preserve contradictions just as easily as it preserves useful context.