Deep dive · Folio III
To find a sentence in a library, you don't read every book: you follow the signs. AI agents are starting to arrange their memory the same way. How much does it really save?
An AI agent that works for a long time builds up memories: conversations, documents, past actions. The simplest approach is to keep everything flat and search the whole pile. But the bigger the pile, the more each search costs.
The alternative: arrange memories in a tree, with summaries at each level. For 1,000 memories, about ten levels are enough; for a million, about twenty. This is called a logarithmic cost: it grows by one step when memory doubles.
A 2026 theory shows that these systems, however different they look, all chain three operations. Extract: cut raw data into units of information. Group: form groups and give each one a representative, often a summary. Traverse: for a given question and word budget, choose what to read.
The key notion is the representative's self-sufficiency: if the summary is enough to answer, you stop there; otherwise, you have to go down to the details. The authors show that 11 existing systems fit this same framework.
Source: Talebirad et al., Toward a Theory of Hierarchical Memory (2026)
Fig. E — Read everything or go down the tree
Read everything: 16 steps
Go down the tree: 4 steps
16 documents · the tree is faster by a factor of 4
Illustrative diagram: it shows the idea, not measurements.
Established
Inspired by operating systems: a small, fast memory (the model's context) and a large, slow memory (an external database). The model moves information between them itself, through function calls, like a computer swapping pages between RAM and disk.
Source: Packer et al., MemGPT: Towards LLMs as Operating Systems (2023)
Promising
Each node summarises what lies below it, with different levels of abstraction depending on depth. Each new piece of information enters at the root and moves down to its place. In long conversations it outperforms MemGPT; on multi-document questions it approaches methods that prepare everything in advance and beats GraphRAG.
Source: Rezazadeh et al., MemTree: Dynamic Tree Memory (2024)
Promising
Memories are arranged like folders and subfolders; each summary keeps a link to the raw memory. A small agent, trained by reinforcement learning, browses the tree to take only what is needed. In long conversations, it uses at most 22% of the words consumed by baseline methods.
Promising
A memory that keeps changing has to be maintained. MemForest arranges memories in time-ordered trees and updates only the paths touched by new information, instead of rewriting every summary. Maintenance cost then follows the size of the change, not the size of the memory.
Source: MemForest: Hierarchical Temporal Indexing for Agent Memory (2026)
The agent memories most used in 2025-2026 are moving away from the strict tree. They extract facts and link them like a researcher's notes. The cost stays low because only the useful notes are read.
That is why we raise H5 ("a graph, not a tree") from 60 to 65%. One reservation, raised by critics: A-MEM's links only say "these two notes are similar", not why. A real hierarchy of knowledge would need links that state the relationship.
H5 · A graph, not a tree (65 %)
Promising
Instead of re-reading the whole conversation, Mem0 extracts the salient information, merges it with what it already knows, then retrieves it on demand (one variant arranges it as a graph). Compared with the "re-read everything" approach, it cuts response time by 91% and token cost by more than 90%, while answering better (+26% relative to OpenAI's memory on the LOCOMO test).
Source: Chhikara et al., Mem0: Scalable Long-Term Memory (2025)
Promising
Each memory becomes a note with keywords, tags and links to related notes. When a new note arrives, it can update the old ones: the memory reorganises itself. Presented at NeurIPS 2025, it beats previous methods with six different models.
Source: Xu et al., A-MEM: Agentic Memory for LLM Agents (NeurIPS 2025)
To be honest, logarithmic cost is an ideal. It assumes a well-balanced tree and reliable summaries. In practice, building the tree costs model calls to write the summaries, and a wrong turn at the top of the tree means missing the information at the bottom. Many systems therefore read several branches at once.
Papers mostly measure words consumed and answer accuracy, rarely the theoretical cost. So we keep the "promising" status: the gain is real and often spectacular, but the word "logarithmic" describes the trend, not a guarantee.
Three weak points keep coming back. Routing: a bad choice early in the descent is hard to recover from. Loss of detail: a summary always throws something away, and sometimes that is the answer. Maintenance: a living memory changes all the time, and reorganising the tree is expensive if done badly.
This work gives substance to H2 on the external-memory side: the "bottom" of our hierarchy can be looked up at a cost that grows very slowly. And it confirms H5: MemTree, GraphRAG and HORMA all end up linking branches together (raw memory, entities, links in time). Pure trees turn into graphs.
H2 · Learn the top, look up the bottom (75 %) H5 · A graph, not a tree (65 %)
Arrange OrigoTech's own articles as a tree (theme, sub-theme, article, paragraph), then measure, for a set of questions, how many words have to be read flat versus going down. Repeat each time the site doubles in size, to see whether the curve really follows the logarithm.