Add memory and continues learning support

[LeoVS09]

It is possible to add continues learning plugin using automated hooks to manage agent memory.

Need find a way properly distribute agent context at session end and when he finds new useful information between:

• Claude.md - long term critical experience
• Rules - findings and patterns
• Skills - complex use cases and strategies

On top of that many current researches and projects incorporate:

• progress.md, state.md or log.md to describe current state.
• Roadmap.md or Main.md to incorporate long term projects roadmap

Possible to use following patterns:

• Git-Context-Controller
• Context Repositories
• Answers-only Memory storage. TODO: find arxiv link, currently only mentioned in Mastra AI research
• Manus memory management

[LeoVS09]

Possible to explore group-agent-evolutation as part of learning process

[cinjoff]

@LeoVS09
I saw your research doc mentioning looking into RAG semantic search & injecting in context before writing.
claude-mem does that. They have good docs & best practices explainer.

I've digested them as reusable shared rules doc, which different skills of mine refer to:
https://github.com/cinjoff/fhhs-skills/blob/main/.claude/skills/shared/claude-mem-rules.md

Hope this helps.

• Someone recently forked that and re-designed it, with a their own thesis of making it lighter, I haven't really explored that yet.
  https://github.com/sdsrss/claude-mem-lite

-- Good job over here btw, inspiring work, i'm exploring incorporating CEK into my own workflows.

[acsezen]

Do you think, offload memory part to honcho/hindsight rather than just claude memory may make sense? Or it will bring some unnecessary third party dependency?

Also, I suggest PI Agent support :)

[LeoVS09]

@cinjoff, thank you for your input and skill. It looks very promising.

I was thinking about using claude-mem. But I was very concerned about how it decides what to remember. Specifically, how it strips useless information and removes outdated. Maybe you can share your experience with it? Have you ever had cases when an agent did something incorrectly and remembered it? And then started recalling it and repeating this issue again? How are you solving/avoiding it?

Do you think, offload memory part to honcho/hindsight rather than just claude memory may make sense?

@acsezen, thank you for sharing honcho/hindsight, impressive solutions. I heard only about supermemory before, and it had an issue with long indexing. Which looks like solved by hindsight. So, probably, yes, it makes it more useful than SQLite-based databases.

But anyway, I believe saving and recalling past session info is a relatively well-solved problem. Many solutions show more than 80% accuracy. But for true continuous learning, it is not enough just to remember past experience. Much more important to learn lessons from it. Plus, many models, even the smartest, frequently hallucinate and make incorrect assumptions/decisions. Which I personally prefer not to waste time correcting, and instead just simply start a new session.

Memory Approaches

I'm aware of 2 types of memory approaches in different papers:

• Saving facts - remembering past conversations, information about people, and work that was done in the form of raw text, graph nodes, and vectors.
• Saving solution strategies - extracting what was done correctly, and what was done wrong, and writing solution strategies that are better to use, and better to avoid.

Saving facts

Saving facts is a more researched area, but I see its applicability for a personal agent or content writer agent. For software development, on the other hand, I do not see how exactly it can be useful. The current up-to-date state of the solution is already present in the code. If an agent needs to do many tasks one by one, it can be solved by a roadmap.md file or something similar, with fewer halicunations. I would be grateful if you could share your experience with such memory solutions. How did you use it? What did it improve?

Solution strategies

Regardless, the situation of saving solution strategies is much more complex. I am not aware of any built-in solution that can do it or provide a proper framework for it. So I experimented with it when we were building SDD, and recently added this to SADD:

• /sdd:plan command has a step that creates new or refines existing skills, applicable to the task. It still does not work exactly as supposed, but the results are already promising. It produces skills that explain specific usage patterns and proper ways to avoid issues when using libraries or writing some features.
• /sadd:do-and-judge has a Judge agent that creates new or modifies existing rules if it finds a repetitive issue during verification. As a result, future agents, when making similar tasks, avoid repeating the same issue again.

Skills and rules are plain Markdown files that are easy to check, modify, or delete if you do not agree with them. This makes agent memory management very simple and allows keeping this memory abstract from specific facts or knowledge that can become obsolete with time.

Feature Target

With the continuous learning feature, I want to see something similar. An agent that is capable of remembering important information and stripping everything else, without requiring too much work from the developer to maintain proper memory. And it will be great to have a meaningful impact on the quality of agent results, which will offset the amount of tokens that the agent will spend on maintaining/recaling his memory.

One direction in which it may be possible to do so is by fully decoupling code writing from planning and orchestration. So there will be:

• An orchestrator agent that remembers all facts and past experiences, and as a result, can provide proper context to all sub-agents
• A planner who knows only about the current task and has knowledge of lessons from previous similar successful tasks.
• Developer-Judge pair that is trying to implement the task and constantly learning from mistakes and each other.

The current implementation of SDD and SADD showed that it might work, but plugins use a quite "simple" implementation that still eats session context and tokens very quickly. Adding more context and decision-making to all agents will definitely eliminate any profits from this approach.