Repository navigation
Participants with local on-disk storage, but without OS page cache flush #774
Description
According to the benchmark rules,
if it's a database with local on-disk storage, the first query should be run after dropping the page cache
The following local disk-based participants do not flush the OS page cache between query runs. This gives them an unfair advantage on repeated queries since data may be served from the OS cache rather than being read from disk.
The corresponding scripts should be fixed to put everyone in the same conditions.
For reference, the correct way to flush the page cache is:
sync && echo 3 | sudo tee /proc/sys/vm/drop_cachesList
Note that the list may be incomplete.
- chdb-dataframe | Reads parquet locally via Python chdb-dataframe: clear page cache between queries #779
- clickhouse-datalake | Uses
clickhouse local, no OS cache flush - clickhouse-datalake-partitioned | Uses
clickhouse local, no OS cache flush - duckdb-dataframe | Reads parquet locally via Python
- elasticsearch | Clears ES query cache only, not OS page cache
- hydra | PostgreSQL-based, no cache flush
- locustdb | Disk-based (RocksDB), no cache flush (benchmark broken)
- mongodb | Local installation, no cache flush
- pandas | Reads parquet locally via Python
- polars | Reads parquet locally via Python
- polars-dataframe | Reads parquet locally via Python
- tembo-olap | PostgreSQL-based, no cache flush
Reacted by Mark, Christian Winter, Nick Woolmer and Robert Schulze
Metadata
Metadata
Assignees
Labels
No labels
Activity
Address #774 in chdb-dataframe
rschu1ze commentedon Feb 4, 2026
Thanks for reporting. I think we can crowdsource fixing this.
I'll start with #779 for chdb-dataframe.
puzpuzpuz commentedon Feb 9, 2026
@rschu1ze how about tagging maintainers for the corresponding benchmarks? Here is the list of projects for which I was able to find the maintainer or, at least, an active contributor:
Note that I didn't tag these persons as I leave it up to you if you'd like to ask them for help.
Adjust scripts for #774
rschu1ze commentedon Feb 9, 2026
Nevermind, I'll make the changes quickly by myself.
Adjust scripts for #774
Adjust scripts for #774
Adjust scripts for #774
puzpuzpuz commentedon Feb 19, 2026
@rschu1ze thanks for handling the issue!