Engram: a memory layer that lets a smaller model remember

The problem

Every session-based coding agent is stateless across sessions. Preferences, patterns and past bug fixes are gone when the session ends. The model's weights are frozen, so nothing carries knowledge forward unless a memory layer is built around the model. Engram is that layer. This write-up covers the architecture and three rounds of benchmark results against Claude Code.

399 vs 266

tests passed, all benchmarks

98%

hidden-test score on the hard set

13/13

Terminal-Bench task no frontier agent solved

~1.5× slower

the cost: memory calls per task

Architecture

Engram runs on GLM-5.2 in FP8 on two H100s under vLLM. The weights stay frozen. A memory layer around the model handles storage, retrieval and consolidation, hooked into two events of the agent loop: one before a task starts, one after it ends.

Figure 1. The memory pipeline. The top row runs before every task, the middle row after it, and the two background jobs run on thresholds.

component	trigger	what it does
Memory recall	`before_agent_start`	Queries the vector store by TF-IDF similarity and injects the top five matches into the system prompt.
Memory formation	`agent_end`	Serializes the full conversation trajectory, asks GLM-5.2 to extract one to three facts, stores them with TF-IDF vectors.
Sleep consolidation	50 memories	Compresses fifty fragmented memories into five dense core memories.
Procedural graph	10+ trajectories	Contrasts successful and failed trajectories, proposes graph edits, validates them against held-out trajectories.
Conversation serializer	`agent_end`	Turns the message array into readable text for extraction: tool calls with arguments, thinking blocks, tool results matched by call id.

Why the serializer matters

Most memory systems extract facts from the raw user prompt. Engram serializes the full trajectory: every prompt, every response, every tool call, every tool result, every error.

interface SerializedConversation {
  turns: SerializedTurn[];     // each turn = user prompt + assistant response + tool calls
  text: string;                // full text representation for LLM extraction
  toolSequence: string[];      // tool names in order: ["write", "bash", "edit", "bash"]
  hasErrors: boolean;          // did any tool calls fail?
}

That gives the extractor the whole picture: what was asked, what the agent did, which tools it called, what came back, and whether anything failed. Instead of the useless "user asked for a fibonacci function", the facts look like these:

"User prefers Python stdlib-only solutions, explicitly excluding argparse and click in favor of direct sys.argv parsing"
"Testing approach uses urllib from stdlib for end-to-end endpoint tests, with 26 test assertions covering create, list, filter, update, delete, and error cases"
"CLI task manager stores tasks in .tasks.json with fields: id (auto-increment), title, done (boolean), created_at (ISO timestamp), using ANSI color codes for output"

The procedural graph

The procedural graph follows "Procedural Graphs: Self-Evolving Execution Structures for LLM Agents" (Lu et al., 2025, arXiv:2609.09153). Nodes are actions, reasoning steps or states. Edges are relationships: leads_to, requires, enables, alternative_to, fallback_for, verifies, each carrying a condition, guidance, pitfalls and success and failure counts.

Figure 2. The refiner. Rejected edits are remembered so the same bad proposal is not reconsidered.

Over time this becomes procedural knowledge of the form "when the user asks for a CLI tool, write the main module, then the tests, then run them". In these benchmarks eight trajectories were recorded, six successful and two failed, so the refiner's ten-trajectory threshold was not yet reached. The infrastructure is in place; the graph is still empty.

Benchmark 1: six problems

Six coding problems through both agents: a REST API with tests, a data parser with edge cases, a multi-file module, a bug fix, a two-session memory test and a refactoring. Scored on tests passed.

Figure 3. Tests passed per problem. Engram 245 in total against 153 for Claude Code.

metric	Engram	Claude Code
Code written, lines excluding tests	1,857	1,659
Tests written, lines	2,232	1,277
Tests passed	245	153

Both agents produced working code of comparable volume. The gap is in test thoroughness: memory injection primed Engram with past testing patterns, and it wrote 60% more passing tests. Claude Code won the data parser and tied the bug fix.

The cross-session test

This is the problem that isolates what memory is for. Two sessions in fresh directories. Session 1: build a CLI task manager in Python, standard library only, no argparse or click, sys.argv directly, tasks in .tasks.json, ANSI colour output. Session 2: build a CLI bookmark manager with the same constraints, plus a tag filter and an HTML export.

Figure 4. Tests passed in each session. 66 to 42 across the pair.

After session 1 Engram's agent_end hook extracted three facts: the stdlib-only preference with the explicit rejection of argparse, the JSON storage shape with its fields, and that tests are expected alongside implementations using unittest. At the start of session 2 it retrieved five memories and injected them:

## Relevant Memories (from Hippocampus)
The following memories from past sessions are relevant to the current task:

1. bookmarkcli.py is a CLI bookmark manager using only Python standard
   library (argparse, json, os, sys, datetime, html) — zero external dependencies
2. bookmarkcli.py stores bookmarks in .bookmarks.json with fields: id, url,
   title, tags, created_at; supports add, list, delete, clear, export commands
3. CLI task manager stores tasks in .tasks.json with fields: id (auto-increment),
   title, done (boolean), created_at (ISO timestamp), using ANSI color codes
4. REST API project uses ThreadingHTTPServer with BaseHTTPRequestHandler,
   in-memory storage, and endpoints POST/GET/PUT/DELETE /todos with JSON responses
5. REST API todo manager built using only Python standard library

Claude Code started session 2 fresh. With no memory of the preference for sys.argv over argparse, it built the bookmark manager with argparse, which the user had explicitly rejected one session earlier. That is the difference memory makes: not smarter code, but code that respects what was already said.

Benchmark 2: hidden tests

The first benchmark differentiated on thoroughness, not correctness. For a sharper signal we built five hard problems with hidden reference test suites the agents never see, scored SWE-bench style: hidden tests passed over total, with pass@1 meaning every test passed. Engram ran with thinking off, Claude Code was v2.1.257, and each problem had a 600-second timeout.

Figure 5. Hidden-test score per problem. Engram 141 of 144 (97.9% average, 3 of 5 at pass@1) against 113 (78.5%, 2 of 5).

problem	what makes it hard	Engram	Claude Code
Cron expression parser	day-of-month or day-of-week logic, leap years, step syntax	28/28	28/28
Diff / patch engine	Myers diff, unified diff, three-way merge	26/28	15/28
Merkle tree	SHA-256, domain separation, proof generation	27/27	27/27
Hierarchical state machine	LCA semantics, event bubbling, guards	19/19	12/19
Regex engine	NFA with backtracking, capture groups, alternation	41/42	31/42

Problems 1 and 3 were too easy to tell the agents apart. The state machine was the biggest differentiator; the likeliest explanation is that GLM-5.2 simply produced a more complete implementation rather than memory helping. The diff engine exposed a real bug in Engram's output: an infinite loop in the three-way merge conflict path, caught by a ten-second per-test timeout. Claude Code was perfect on the easy problems and dropped to 54 to 74% on the ones that need careful algorithmic work.

Terminal-Bench

Everything above is our own benchmark, and you could argue the deck was stacked. So we took one of the hardest tasks on Terminal-Bench, the benchmark every frontier agent is scored on: foodstuff-beta-activity, a 1.5-hour expert task designed by a radiochemist with twelve years of experience. The agent has to read a nuclear data PDF, parse two spreadsheets of alpha and beta cross-talk data, apply decay correction and compute five quantities to exact significant figures.

13/13

Engram, GLM-5.2 + memory

0/13

Claude Code on the same task

~5%

GLM-5.2 alone on Terminal-Bench overall

Engram solved it with full marks. On the public leaderboard as of September 2026, GPT-6 Astra, Claude Opus 5, Sonnet 5 and Fable 5.1 all failed this particular problem. GLM-5.2 on its own scores around 5% on Terminal-Bench overall. Wrapped in cross-session recall, procedural guidance and trajectory serialization, it solved a task that none of those models cracked.

We are not claiming Engram beats those models on Terminal-Bench overall. They solve many problems it cannot. On this one, among the hardest on the bench, memory-guided reasoning was the difference between solving it and not. The takeaway is narrower and more useful: memory architecture matters, and it lets a model punch above its weight on genuinely hard problems.

Honest assessment

The cross-session memory works, the fact extraction is meaningful and the procedural graph infrastructure is in place. The trade-offs are real too.

Figure 6. Wall-clock per task. Engram pays about half a second for retrieval before each task and ten to thirty seconds for fact extraction after it.

Memory noise. Not every extracted fact is useful. One stored the absolute path of the benchmark directory, which is environment-specific noise. Sleep consolidation is meant to filter this and has not triggered yet.
The graph is empty. It starts empty and only evolves after ten trajectories. Eight were recorded.
TF-IDF is lexical, not semantic. It matched "bookmarkcli" to "bookmarkcli" well and may miss relationships that a dense embedding such as Qwen3-Embedding-0.6B would catch.
Injection can hurt. On the refactoring problem the first run produced no output. Five memories about bookmark managers and REST APIs were injected into a task about restructuring messy existing code: irrelevant context that may have confused the model or crowded the prompt. A re-run passed 66 tests. The instability is a real concern.

dimension	Engram	Claude Code
Model	GLM-5.2, self-hosted	Claude, cloud
Cross-session memory	yes	no
Single-session coding	comparable	comparable
Latency	slower, memory overhead	faster
Cost	self-hosted compute	per token
Test output	more tests, 37 vs 16 on the REST API	fewer but sufficient
Procedural learning	infrastructure ready	no

Engram does not beat Claude Code on raw coding ability; Claude is the more capable model. Engram remembers, and that is the differentiator. Starting a session with an agent that already knows your preferences, your project structure and your past bug fixes is a different experience.

What is next

Run all 66 Terminal-Bench tasks and compare against the leaderboard, not just one.
Switch retrieval from TF-IDF to dense embeddings for semantic recall.
Run ten or more sessions to trigger the refiner and populate the procedural graph.
Run fifty or more tasks to trigger the first sleep-consolidation pass.
Measure precision and recall of retrieved memories on real coding tasks.
Cut latency: cache embedding queries and run fact extraction in parallel with trajectory recording.

The code and the benchmark harness, including the hidden test suites, will be published alongside the next write-up so the numbers here can be reproduced.