I'm on the hunt for a Python library that can handle the storage of messages in a database in a structured and queryable way. I also need it to maintain rolling summaries of conversations to help assemble context for my LLM calls. However, I'm not interested in full agent frameworks (like Letta or LangChain agents) or memory systems that focus on fact extraction and semantic retrieval. I've explored Mem0, but it seems more like a memory layer geared towards fact extraction rather than straightforward storage and summarization. I found MemexLLM, but it doesn't seem well-maintained either. Is there any library that meets my needs cleanly, or does everyone end up building their own solutions?
3 Answers
Honestly, you’re not missing much in this area—most available libraries either go full agent framework or are focused on memory extraction. The combination of clean storage and effective summarization is surprisingly underdeveloped in this space. People usually end up rolling their own solutions instead.
I haven't found the perfect library either. I ended up creating my own solution using SQLAlchemy for message storage, Pydantic for serialization, and a function to summarize every N messages. It didn't take long, and I have full control over the schema. Once the active messages go over a threshold, I call the LLM to summarize older messages, store them, and tidy up the context assembly. It’s smooth to implement, especially with FastAPI for async handling.
That sounds like a solid plan! I’m thinking I might go that route too. Manual control over the process seems worth it.
For my use case, I rolled my own solution with a messages table that includes session_id, role, content, and timestamp. When assembling context, I fetch the last N messages and a cached summary of older ones. In the end, it took about 150 lines of code, and I own the entire setup. If you’re looking for something pre-built, you might give Mem0 a shot; it’s lighter than the full agent frameworks and covers storage plus rolling summaries without the added complexity. Definitely worth checking out before committing to build your own.
Right? It feels like such a necessary feature, but it's hard to find anything that fits those criteria without the extra bloat.