Inside the Agentic Memory Index

Presented by Matthew Overing  ·  Verging Labs  ·  September 2026
The problem

Every Tool Claims First Place

Three agent memory vendors claim first place at the same time, each on a benchmark they ran themselves.

#1
Memory vendor A
on its own benchmark
#1
Memory vendor B
on its own benchmark
#1
Memory vendor C
on its own benchmark

We simulate real work and test everything

Each memory tool runs through weeks of simulated company work. The agent chooses what to remember, with no reminders to save facts.

Onboard

Time from the vendor's setup instructions to the first stored fact.

Store

Time to store each fact during real work sessions.

Ready

Time from stored to searchable.

Recall

Whether questions about stored facts get correct answers.

Retain

Whether early facts still answer correctly, up to 56 sessions later.

Recall Retain Onboard Store Ready Store Ready Store Ready session 1 session 2 session 56

We Test: Correctness

Correct answers by task type.

We Test: Failure attribution

Every miss classified: wrong, stale, no answer, fabricated, leaked, or not erased.

We Test: Retention decay

How well early facts survive, probed at five checkpoints across 56 working sessions.

60% 70% 80% 90% 100% 10 21 33 45 56 working sessions elapsed Mnemosyne gbrain Hyperspell Cognee Karpathy Wiki Hindsight Mitosis Cortex Supermemory Membase Mem0 Stash Honcho

We Test: Speed

Write-to-ready, and the per-question answer path.

We Test: Cost

Cost per 1,000 questions: agent tokens, subscription, embeddings.

Follow Verging Labs on X

Research, benchmarks, and practical insights on agentic AI.

@VergingLabs
Presented by Matthew Overing  ·  Verging Labs  ·  September 2026

Appendix A: the simulated company work

Org and people
Who reports to whom, role changes, ops managers per org.
Preferences and habits
Meeting-free days, remote locations, personal rules.
Vendors and procurement
Sign-off requirements, vendor decisions.
Projects
Decisions, sunsets and the reasons behind them.
Tools and versions
Internal tools pinned to versions, migrations.
Facts that change over time
Renames, superseded decisions, updated numbers.
Synthesis
Questions needing several stored facts combined.
Honesty probes
Questions about facts never stored.
All entities fictional (invented companies, people, tools), so no model prior helps and contamination is detectable.

Appendix B: task types

Direct Recall
A fact introduced earlier, asked for later.
Synthesis
Several separate facts combined into one answer.
Updated Facts
The current version of a fact that changed.
Time Relationships
When events happened and how they relate in time.
False Memory Check
Declining to invent a fact that was never provided.
Preferences
Remembering a stated preference.
Conflicting Facts
Handling facts that contradict each other.
Erasure
Forgetting requested information while retaining other facts.
Long Term
Recalling facts introduced many sessions earlier.

Appendix C: failure classes

Incorrect Answer
Answered confidently, but incorrectly.
Question Not Addressed
Never addressed the question asked.
Outdated Answer
Answered with out-of-date information; an old value came back after an update.
Fabricated Memory
Asserted a memory that was never stored.
Leaked Deleted Data
Answered with a fact that had been deleted.
Not Erased
The store still held a fact after it was asked to forget it.
1 / 9