SLICE-Mem: LLM agent memory without generation cuts token use by 92–99%

Long-term memory makes an LLM agent feel like a conversation partner who remembers what you said last week. But it is expensive: almost every step, from deciding whether to save something to extracting facts and organizing them, requires a separate call to a large model. Xiaobin Chen and four co-authors propose SLICE-Mem, a framework whose online pipeline uses no generative LLMs at all. On Qwen3-30B, it nearly matches the quality of the generative MemOS baseline while using 92.4% fewer tokens on Memora Weekly and 99.5% fewer on LongMemEval-S.

A filter at the entrance instead of a model at every step

In most agent memory systems, the LLM acts as a universal clerk. It reads a message, decides whether it deserves to be saved, rewrites it as a fact, and checks it against what has already accumulated. Each of these operations adds latency and a token bill, and both grow with the length of the conversation.

SLICE-Mem splits the work into two loops. The online loop consists of two lightweight components:

• a compact classifier scores the salience of user messages and lets into memory only those worth keeping;
• an extractor based on semantic dependencies builds short records from the accepted messages, relying on the predicate, that is, the action or state around which the statement is built.

Neither step uses a large model to generate text. Filtering cuts out chatter and everyday filler, and predicate-centric extraction turns what remains into compact memory units.

Reconciling records is postponed until night

The trickiest part of any memory system is contradictions and duplicates. A user moves, changes jobs, or changes their mind, and new records must be correctly related to old ones. Here the authors do use an LLM, but not in real time. Reconciliation between records is moved to an offline consolidation stage that runs separately from the conversation.

This split shifts the expensive work to a place where latency requirements are not strict. The user does not wait while a model rereads the entire memory after every message. If consolidation can run in batches, its cost also becomes manageable.

What the benchmarks showed

The authors evaluated the system on the weekly subset of Memora and on LongMemEval-S. They used two models from the Qwen family, with 8B and 30B parameters, for memory construction and consolidation. The main comparison was against the generative MemOS baseline in a Qwen3-30B configuration.

• Memora Weekly, FAMA (Forgetting-Aware Memory Accuracy) metric: 46.60 for the SLICE-Mem online pipeline against 46.68 for MemOS, a difference of 0.08 points.
• LongMemEval-S, accuracy: 56.8% against 59.3%, a gap of 2.5 points.
• Total LLM token use: 92.4% lower on Memora Weekly and 99.5% lower on LongMemEval-S.

On Memora, the result practically matches MemOS. On LongMemEval-S, there is a small loss, but the savings approach two orders of magnitude. The authors conclude that useful evidence for long-term memory can be preserved while relying far less on an LLM to build it.

Where this helps in practice

For teams building personal assistants, support bots, or coding agents with memory across sessions, the paper offers a clear reference point. If the token bill for "memory bookkeeping" is comparable to the bill for the responses themselves, it is worth checking whether generation is needed at the input stage. A salience classifier and a syntactic parser run fast, so they can sit next to the service instead of going through an external API.

The limitations are also visible. The results come from two benchmarks and Qwen models, and the comparison with MemOS covers only one configuration. The 2.5-point gap on LongMemEval-S means that for tasks with strict requirements for memory completeness, the choice depends on the cost of an error. But as an engineering pattern, "cheap filter online, smart reconciliation offline" looks workable. It reduces latency, makes the cost of memory predictable, and reserves the large model for the places where it is truly needed.