I'm considering building an application that adds persistent memory to AI chats. Imagine working with an AI on a software project: you define the architecture, make design decisions, fix bugs, and leave some tasks unfinished. A few minutes later, the AI may forget an earlier decision and suggest something that conflicts with the existing work.
Instead of relying only on the conversation history, a separate memory layer could track the current objective, user requirements, important decisions, project state, completed and pending tasks, dependencies, and other durable facts. When the agent needs context, the system would retrieve the most relevant memories rather than sending the entire conversation every time.
Could this work as a general-purpose memory architecture for long-running AI agents? I'm especially interested in approaches involving LLM agents, retrieval-augmented generation, vector databases, knowledge graphs, project documentation, or automatic extraction of decisions and task state.
3 Answers
A practical solution is to keep structured project records such as architecture decision records, notes, task lists, and development logs. These are more reliable than asking the model to infer everything from an old conversation, and they make the project state explicit and reviewable.
This is already an active area, with several memory-layer and personal knowledge-base tools experimenting with summaries, semantic retrieval, and persistent project context. It’s worth comparing existing systems before building one from scratch, especially how they decide what information to save and when an old memory should be updated or removed.
You can also instruct the AI to maintain a project document containing the requirements, decisions, current status, and next steps. Keeping that document concise and structured makes it easier to include in future prompts than replaying the entire chat.
That makes sense, and I agree that documentation solves part of the problem. I’m mainly interested in automating the maintenance layer so the system can extract decisions, dependencies, and task changes from the interaction instead of relying entirely on the developer or agent to update every record manually.