Minimal RAG Memory Matches SOTA on Benchmarks but Fails Adversarial Probes
arXiv — cs.AI · Jingyu Liu, Zongze Li, Zach Xu, Zhanhui Zhou, Tahseen Rabbani, Dawn Song, Ce Zhang · Published on Readzer
Summary
The paper introduces BRIDGE, a minimal RAG memory system that achieves parity with prior state-of-the-art on LongMemEval-S and BEAM, and outperforms the best baseline on EverMemBench by 5.7-6.3 points. However, on five new adversarial probes testing rule and pattern application, no architecture wins universally: atomic-fact extractors excel at rule application, while BRIDGE's block-summary memory wins at pattern induction. The authors argue that standard benchmarks measure retrieval, not what memory enables an agent to do, and propose evaluating agentic memory on action-oriented tasks.
This is a reviewed summary. Read the complete article at the original source.
Why it matters by industry
- Artificial intelligence
This research challenges current evaluation methods for agentic memory, showing that minimal systems can match complex ones on standard benchmarks, but fail on tasks requiring action based on remembered rules. It highlights the need for new evaluation paradigms in AI agent design.