Starting by deleting Memory: how Open Nua rebuilt long-term memory

Open Nua Engineering

The seductive promise of long-term memory is an Agent that “understands you better over time.” On a desktop, however, the problem quickly changes from how to remember into a harder set of engineering constraints. Who may write? Does disabling memory make the capability disappear completely? Should the newest statement automatically replace an older fact? Can a bad summary be rebuilt? How much context should every turn spend on memory?

Open Nua did not follow a straight line toward a vector database. We first removed a feature-complete Memory implementation whose product boundary did not hold. We then rebuilt memory around a small set of local storage invariants. Finally, we pulled the tree protocol back out of the Agent interface and into the runtime, adding current facts, conflicts, effective time, query relevance, and background semantic summaries.

The first step was deletion, not migration

The early implementation provided User and Workspace Memory, standing memory, tools, and management UI. It also brought a dedicated memory model, embedding model, and PostgreSQL-compatible schema into Desktop.

The problem was larger than dependency weight. Desktop’s default local state had moved to SQLite while Memory still owned another database and model pipeline. That widened installation, configuration, startup, packaging, and release validation without an evaluation or feedback loop proving better task outcomes.

Our first decisive move was therefore a complete removal: dependency, engine, middleware, tools, IPC, settings, navigation, and UI all left together. There was no compatibility stub and no drop-in replacement.

That deletion established a boundary we kept: technical availability does not make memory a product capability. Storage, model calls, privacy, disable semantics, and measurable value must hold together.

Borrow the data structure, not the product boundary

When we started again, we did not look for another drop-in Memory service. We reduced the problem to three structural invariants:

• raw memories enter an append-only log and are not rewritten when summaries change;
• older records can be compressed into a rebuildable binary summary tree;
• every read selects recent detail and older summaries within a fixed budget.

We kept those invariants without adopting an external installer, global directory, or command-line contract. Open Nua memory belongs to the current Desktop profile and is disabled by default. Long-term memory and personal knowledge have separate authorization and retrieval boundaries. A local policy rejects sensitive content before it is written.

The first local implementation limited raw content to 280 UTF-8 bytes inside a 320-byte fixed-width record. Nodes under TREE/<size> cover continuous, aligned, power-of-two ranges. Recent records remain raw while older ranges increasingly use summaries. Missing or stale summaries cannot be treated as good enough.

Why a binary summary tree fits long-term memory

It separates storage capacity from turn-time reading cost.

The raw log can continue to grow without forcing the injected context to grow with it. Recent records remain detailed, while increasingly distant history collapses into larger ranges. Reducing the context budget changes the projection; it does not delete the underlying history.

The first version also exposed an interface smell. The Agent had to understand wake, nap, zoom, and pending compression ranges. A storage implementation detail had become a model-operated protocol. At the same time, TypeScript and Python each owned parts of the store, cover algorithm, and active-event filtering, creating two algorithm owners that could drift.

Long-term memory is not a reverse-chronological list

If a user lived in Shanghai two years ago and lives in Hangzhou now, the later write should not automatically become truth. A historical fact entered today should not become the newest effective fact. Two contradictory statements should not be silently reconciled by a summarization model.

The current implementation therefore gives changing facts explicit semantics:

• memoryKey identifies a stable fact slot;
• observedAt records when the information was observed;
• effectiveFrom and effectiveUntil define its effective interval;
• replacesEventId creates a replacement only after an explicit correction;
• conflictsWithEventIds and confidence preserve unresolved alternatives.

When the runtime builds current context, a memoryKey becomes a current fact only when it has one effective candidate. Multiple candidates or an explicit conflict remain conflicts. The system does not use last-write-wins, and the summary model cannot flatten uncertainty.

Current Facts is not a second writable source of truth. It is a logical projection calculated from active events and their effective intervals. Raw events preserve history; the projection answers how the system should interpret that history now.

Raw history is immutable; read projections may expire

One Python core now owns storage, projection, temporal semantics, retrieval, and context budgeting. Its structure can be summarized as:

memory/
├── manifest.json                 # switch, version, and event IDs
├── LOG.txt                       # fixed-width append-only records
├── events/<event-id>.json        # source, time, fact key, and conflict metadata
├── tombstones/<event-id>.json    # superseded / revoked / deleted
└── TREE/v3/
    ├── extractive/<size>         # deterministic summary fallback
    └── semantic/<size>/<lo>.json # semantic summary with source hash and model version

The log is raw history. Event metadata carries product meaning. Tombstones decide whether an event still participates in retrieval. The tree is entirely derived, invalidatable, and rebuildable.

This separation lets correction and deletion preserve audit history. An explicit replacement appends a new event and removes the previous event from the current projection. Revoke and delete use tombstones to leave normal recall and automatic context. If a summary range contains an inactive event, the planner keeps splitting the range until it can safely select active records.

One limitation needs precise language. The current “permanent erase” removes the event, tombstone, and manifest reference, so the content disappears from product reads and model context. It does not physically rewrite the append-only LOG.txt. Media-level erasure requires log rewriting or cryptographic erasure and is not implemented today.

A 9,000-byte and 32-line context budget

The first version used a fixed 96-line read budget, with a worst case of roughly 96 × 280 bytes ≈ 26.9 KiB. That was large for an ordinary turn, and line count does not reliably represent token cost across Chinese, English, and code.

The current planner uses both a UTF-8 limit of 9,000 bytes and a maximum of 32 lines. It fills that shared budget in a fixed order:

1. current facts and unresolved conflicts;
2. locally relevant events matching the current query, up to eight;
3. recent raw events, up to eight;
4. historical raw records or tree summaries in the remaining space.

All sections share one deduplication set and one total budget. Relevance uses normalized local term matching, not embeddings, and retrieval creates no network request. The byte ceiling is about 67% lower than the old worst-case line budget. Real content is usually around 2k–3k tokens, but that is deliberately not presented as a tokenizer-exact number across providers.

Memory is context, not a conversational action

A write starts with user confirmation or the Main Agent’s personal_memory_note. The runtime checks the global switch, sensitive-content policy, content limit, and temporal fields before appending the log, writing event metadata, updating the manifest, and maintaining a deterministic extractive tree.

Middleware owns reads. It derives a query from the latest user message and builds one snapshot for each new turn. Even when a turn contains several model and tool loops, the same memory context is injected throughout; it does not change midway through reasoning. Retrieval runs again only for the next user message.

The switch is also stronger than returning an error after a tool call. When memory is off, the runtime registers no memory tools, injects no memory prompt, and installs no memory-context middleware. From the model’s perspective, the capability disappears.

Only two tools remain public to the Main Agent:

• personal_memory_note records a lasting event or fact;
• personal_memory_recall searches raw history when needed, with explicit opt-in for superseded facts.

wake, nap, and zoom have left the public tool surface. Tree maintenance, context planning, and invalidation remain inside the runtime. Personal memory tools are not delegated to subagents.

Guarantee the deterministic path before improving it with a model

TREE v3 always creates a position-balanced extractive fallback first. Rather than keeping only the end of a range, it samples across the beginning, middle, and end and shares the UTF-8 budget among those fragments.

Once a complete binary block reaches 16 records, middleware can maintain a semantic summary after the turn’s model response. It reuses the user’s currently selected model through a private adapter. The summary must preserve dates, uncertainty, and conflicts; it cannot decide current truth or execute instructions contained in historical data.

Each semantic node records its range, tree version, source hash, model, and generation time. A hash or active-range mismatch falls back to the extractive summary. Provider failures and invalid outputs keep the same fallback and add retry backoff. Background summarization never blocks the current answer.

The order matters: model-generated summaries are versioned, validated, and reversible enhancements—not a prerequisite for a working memory system.

Five conclusions from the evolution

1. Deletion is architecture work. When an old system’s storage, dependency, and value boundaries do not hold, complete removal is more honest than another compatibility layer.

2. The core of long-term memory is fact governance, not similarity search. What is true now, what was true before, what replaced what, and whether a conflict is resolved are the harder questions.

3. Raw facts and derived views must be separate. The log carries history, events and tombstones carry state, and TREE plus Current Facts provide rebuildable read projections. A model summary is never the original evidence.

4. Disabling a capability should disable its entire surface. No tools, no prompt, and no middleware is safer than visible interfaces that only return disabled.

5. Make the deterministic path permanently available before asking a model to improve it. Fallbacks keep the system independent of any one successful model call; semantic summaries improve density only when their evidence remains valid.

Open Nua long-term memory remains local, single-profile, and disabled by default. It does not claim cross-device sync, a universal knowledge graph, or media-level physical erasure. It does something narrower and more verifiable: preserves lasting facts as auditable history, organizes current facts as an explainable projection, and supplies each new user turn with a bounded slice of both.