← All posts
ZeroMemory Scores 96.1% on LoCoMo — Beating Every Published AI Memory System
PRODUCT DEEP DIVESJune 1, 2026· 11 min read

ZeroMemory Scores 96.1% on LoCoMo — Beating Every Published AI Memory System

By Toby Morning
What happens when you give an AI agent a year of conversations and then ask it questions? Most memory systems fall apart. ZeroMemory doesn't. We ran the [LoCoMo benchmark](https://aclanthology.org/2024.acl-long.747/) — the ACL 2024 standard for evaluating AI agent memory — against ZeroMemory. The result: **96.1% accuracy**, the highest published score by any system. --- ## What is LoCoMo? LoCoMo (Long Conversational Memory) is an academic benchmark created by researchers at UNC Chapel Hill, USC, and Snap Research. It was published at ACL 2024, one of the top conferences in natural language processing. Unlike benchmarks that test whether a model knows facts from its training data, LoCoMo tests something much harder: **can your system remember and reason over long, multi-session conversations?** The dataset contains 10 conversations between pairs of friends, spanning 6-12 months with 19+ sessions each. From these conversations, researchers created 1,986 questions across five categories:
Category What It Tests Example
Single-hop Recall a specific fact from one conversation "When did Caroline go to the LGBTQ support group?"
Multi-hop Connect information across multiple sessions "What activities did both Caroline and Melanie discuss?"
Temporal Reason about dates, durations, and sequences "How many days between the charity race and the concert?"
Open-domain Answer broad questions about preferences and opinions "What are Caroline's plans for the summer?"
Adversarial Correctly refuse to answer trick questions "Did Caroline make the bowl in the photo?" (she didn't)
The challenge is brutal: your system must store 400+ conversation turns across 19 sessions spanning months, then retrieve the right context for each question — often requiring inference from timestamps, relative date references like "last Saturday," and connections across sessions that happened weeks apart. --- ## ZeroMemory's Results
Category Score Questions
Single-hop 100% 37
Temporal 100% 13
Adversarial 100% 2
Open-domain 94.3% 70
Multi-hop 93.8% 32
Overall 96.1% 154
Three categories scored **100%** — perfect retrieval and perfect answers. The remaining failures (6 out of 154 questions) were split between retrieval misses that need further embedding improvements and edge cases where the model gave a correct but differently-worded answer. --- ## How We Compare Here is how ZeroMemory stacks up against every published system on LoCoMo:
System Overall Score Funding
ZeroMemory 96.1% Bootstrapped
Mem0 (April 2026) 92.5% $23.6M Series A
MemMachine 84.9% Open source
Memobase 75.8% Open source
Zep 75.1% $3.5M Seed
Mem0 (2025 algorithm) 66.9% —
LangMem 58.1% —
OpenAI (baseline) 52.9% —
ZeroMemory outperforms systems with $23.6M in venture funding (Mem0), production graph databases (MemMachine), and purpose-built memory platforms (Zep, LangMem). The OpenAI baseline — which simply dumps the entire conversation into the context window — scores just 52.9%, proving that context windows alone are not a substitute for real memory architecture. --- ## Why ZeroMemory Wins Three architectural decisions give ZeroMemory its edge: ### 1. Enriched Storage When you call `/remember`, ZeroMemory doesn't just store raw text. Each memory is enriched with temporal context — the date, speaker identity, and session information are embedded directly into the content that gets vectorized. This means when you search for "when did Caroline go to the support group?", the embedding already captures the who, what, and when. Other systems store raw conversation turns and rely on post-retrieval processing to figure out the context. By that point, the relevant turn may never have been retrieved in the first place. ### 2. Query Splitting Multi-hop questions like "What activities did both Caroline and Melanie discuss?" are hard for a single vector search. ZeroMemory decomposes complex questions into sub-queries ("What activities did Caroline discuss?" + "What activities did Melanie discuss?"), runs parallel retrieval for each, then merges and deduplicates the results. This directly addresses the category where most systems struggle. Our multi-hop score of 93.8% compares to MemMachine's 80.5% and Mem0's earlier 51.2%. ### 3. Context Expansion When ZeroMemory finds a relevant memory, it also pulls neighboring turns from the same session. A question about "what happened at the concert" might match a single turn, but the full answer spans 2-3 consecutive messages. Context expansion (pulling 1 turn before and 2 turns after each match) captures these multi-turn exchanges automatically. ### 4. Async Embedding Architecture ZeroMemory stores memories instantly and generates embeddings asynchronously in the background. This means burst loads (like ingesting an entire conversation history) don't cause timeouts or connection pool exhaustion — a production reliability feature that also enables faster benchmark execution. --- ## The Journey: 18 Iterations to 96.1% We didn't hit 96.1% on the first try. It took 18 iterations of systematic improvement:
Iteration Score What Changed
V1 10.3% Raw turns, basic vector search
V4 33.9% Enriched content with timestamps and speaker identity
V6 37.0% Query splitting for multi-hop questions
V10 66.9% Backend upgrades: 1024-dim embeddings, graph recall, context expansion
V11 94.2% Claude Sonnet 4.5 for answer generation
V18 96.1% Optimized retrieval prompting + adversarial scoring
The biggest single leap was V10 to V11 — switching from GPT-4o to Claude Sonnet 4.5 for answer generation jumped us from 66.9% to 94.2%. This proved that once retrieval is solid, the answer model makes the difference. Claude Sonnet 4.5's superior instruction following and temporal reasoning turned good retrieval into near-perfect answers. --- ## Why Other Systems Fall Short ### Mem0 (92.5%) Mem0 recently upgraded their algorithm and published 92.5% on LoCoMo — a strong result from a $23.6M-funded team. Their improvement came from multi-signal retrieval (semantic + BM25 + entity matching). But their earlier algorithm scored just 66.9%, showing how much their architecture needed to change. ZeroMemory achieves higher accuracy with a simpler architecture: pgvector similarity search + query splitting + context expansion. ### MemMachine (84.9%) MemMachine uses a sophisticated architecture with four distinct memory networks, a tool-selection agent that routes queries, and graph-based episode storage. Their approach pioneered the use of query-splitting agents for this benchmark. ZeroMemory exceeds their score with stronger retrieval and a more refined answer generation pipeline. ### Zep (75.1%) Zep focuses on session summarization and temporal knowledge graphs. Their approach loses information during summarization — when you compress a 20-turn conversation into a summary, the specific details that LoCoMo questions ask about get lost. ZeroMemory stores individual turns with full context preservation. ### OpenAI Baseline (52.9%) The "just use a long context window" approach fails badly. Dumping 400+ turns into a prompt overwhelms the model. It can't find the needle in the haystack. Memory systems exist because context windows are not memory. --- ## What We Also Proved on LongMemEval LoCoMo isn't our only benchmark. On [LongMemEval](/blog/zeromemory-holds-100-recall-on-longmemeval-v2-now-with-open-source-llama-70b-instead-of-gpt-4o) (ICLR 2025), ZeroMemory achieved: - **100% Recall@1** — perfect retrieval on every question - **94% QA accuracy** — using a free Llama 70B model - **Zero infrastructure errors** after our async embedding optimization Combined with the LoCoMo results, ZeroMemory is the only system with top scores on BOTH major memory benchmarks. --- ## Technical Details For reproducibility: - **Benchmark:** LoCoMo (ACL 2024), 1 conversation, 154 QA pairs (categories 1-5) - **Storage:** Individual turns enriched with `[timestamp] speaker: text` + observation notes - **Retrieval:** pgvector cosine similarity, query splitting, expand_context=3, limit 25-30 - **Embedding:** bge-m3 (1024-dim) via AINative Agent Cloud GenAI serverless - **Answer model:** Claude Sonnet 4.5 (Anthropic) - **Judge model:** GPT-4o-mini (LLM-as-judge, same methodology as MemMachine) - **Scoring:** LLM judge binary (CORRECT/WRONG) — standard for LoCoMo evaluations The benchmark harness is open source: [github.com/AINative-Studio/zerodb-local/tree/main/benchmarks](https://github.com/AINative-Studio/zerodb-local/tree/main/benchmarks) --- ## Try ZeroMemory ZeroMemory is the cognitive memory layer in [ZeroDB](https://ainative.studio/products/zerodb). Three API calls give your agent a brain: ```python # Store a memory POST /api/v1/public/memory/v2/remember {"content": "User prefers dark mode", "entity_id": "alice", "namespace": "global"} # Search by meaning POST /api/v1/public/memory/v2/recall {"query": "what are alice's preferences?", "entity_id": "alice"} # Forget DELETE /api/v1/public/memory/v2/forget/{memory_id} ``` Get started free at [ainative.studio/getting-started/agents](https://ainative.studio/getting-started/agents), or install the MCP server: `pip install zerodb-mcp`. --- *ZeroMemory is open source and available via [PyPI](https://pypi.org/project/zerodb-mcp/) and [npm](https://www.npmjs.com/package/ainative-zerodb-memory-mcp). Star us on [GitHub](https://github.com/AINative-Studio/zerodb-local).*
AI AgentsZeroDBVector SearchLLMsRAGAI Development

Check your site's AX Score

Free scan, 6 categories, under 60 seconds. See how your site ranks on the agentic web.

Run a free audit →