You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix: README benchmark scores in News section don't match Performance table (LoCoMo/LongMemEval) #2090
I have searched existing issues and this hasn't been mentioned before | 我已搜索现有问题,确认此问题尚未被提及
I have read the project documentation and confirmed this issue doesn't already exist | 我已阅读项目文档并确认此问题尚未存在
This issue is specific to MemOS and not a general software issue | 该问题是针对 MemOS 的,而不是一般软件问题
Bug Description | 问题描述
The News section and the 📊 Performance section of the README report different scores for the same two benchmarks (LoCoMo and LongMemEval), with no explanation for the discrepancy. Readers see two different "current" numbers for the same benchmark on the same page. This affects both README.md and README_ZH.md identically.
News section (README.md lines 49-50):
2026-07-02 · 🏆 MemOS Advances Agent and User Memory Benchmarks
With MemOS, OpenClaw improves average task completion from 36.63% to 50.87% across five agent tasks. MemOS also achieves 92.34 on LoCoMo and 93.40 on LongMemEval, and leads in OmniMemEval...
Performance table (README.md lines 66-77), a few lines below:
Benchmark
Score
LoCoMo
88.83
LongMemEval
89.20
Same benchmarks, same OmniMemEval framework referenced, but the numbers don't match (92.34 vs 88.83, 93.40 vs 89.20).
Root cause (from git history): the News entry was added in #2019 with 92.34/93.40. Later, #2078 rewrote the README and introduced the "Performance" table with 88.83/89.20 ("10 benchmark scores from the 2026-07-02 evaluation report" per the commit message) but didn't update the pre-existing News entry to match, leaving both numbers live on main at the same time.
How to Reproduce | 如何重现
Open README.md (or README_ZH.md) on main.
Compare the LoCoMo/LongMemEval numbers quoted in the News section vs. the 📊 Performance table.
Note the mismatch: 92.34 vs 88.83 (LoCoMo), 93.40 vs 89.20 (LongMemEval).
Environment | 环境信息
N/A — documentation-only issue, not version/platform specific.
Additional Context | 其他信息
Suggested fix: align the News entry with the Performance table (88.83 / 89.20), since the table is the more complete, explicitly-sourced number (10 benchmarks, linked to https://github.com/MemTensor/OmniMemEval). Will open a PR updating both README.md and README_ZH.md.
Summary: Fixed the README score mismatch (LoCoMo 92.34→88.83, LongMemEval 93.40→89.20) in both README.md and README_ZH.md, aligning the News section with the Performance table. Committed on the working branch (64cddbfb) and pushed to origin. Task spec archived to the sibling memos-autodev-specs repo (main @ d8af691).
Now submitting the completion envelope so the scheduler can open the PR.
Activity
Memtensor-AI commentedon Jul 9, 2026
🤖 AutoDev has picked up this issue and started working on it.
Task ID:
a7f814f57a1585cfWorking branch:
bugfix/autodev-2090-20260709093247057Target branch:
latest dev* branchWorkflow: opsp (analysis → coding → testing → PR)
I will post the PR link here once done. If I need more information, I will ask in the comments.
fix(readme): align News section LoCoMo/LongMemEval scores with Perfor…
Memtensor-AI commentedon Jul 9, 2026
✅ AutoDev task
a7f814f57a1585cfcompleted.Summary: Fixed the README score mismatch (LoCoMo 92.34→88.83, LongMemEval 93.40→89.20) in both
README.mdandREADME_ZH.md, aligning the News section with the Performance table. Committed on the working branch (64cddbfb) and pushed toorigin. Task spec archived to the siblingmemos-autodev-specsrepo (main@d8af691).Now submitting the completion envelope so the scheduler can open the PR.
Sources:
Base branch:
mainBranch:
bugfix/autodev-2090-20260709093247057PR: #2092
Assigned to: @CarltonXiang
Reviewers: @MatthewZhuang, @CarltonXiang, @syzsunshine219, @World-controller
Fix #2090: fix: README benchmark scores in News section don't match P…