Skip to content

cache-keys.json grows without bound — 94% dead entries here, and recorded_at_ms is written but never read #224

Closed

Description

@iceteaSA

cache-keys.json memoizes the artifact cache key per project root, so configure doesn't have to shell out to git rev-list on every bind. Entries go in; nothing ever takes them out.

On this box:

total keys:     437
/tmp fixtures:  407
non-/tmp:        30
DEAD (root no longer exists):  411  (94%)
file size:      71 KB

94% of the registry points at directories that don't exist. 407 of them are /tmp/.tmpXXXXXX roots created by aft's own integration tests — every test that configures a temp root leaves a permanent entry in the user's real storage dir.

Why it never shrinks

record_artifact_cache_key_memo (search_index.rs:4121) inserts unconditionally:

entries.insert(
    memo_root_key.to_string(),
    ArtifactCacheKeyMemoEntry {
        key: key.to_string(),
        git_root_commit: git_root_commit.to_string(),
        recorded_at_ms: current_time_millis(),
    },
);
write_artifact_cache_key_memo_file(storage_root, entries)

The read path (read_artifact_cache_key_memo_file, :4148) filters only malformed entries — empty root, bad key shape, empty commit. A well-formed entry whose root was deleted years ago passes the filter forever.

The giveaway that eviction was intended: recorded_at_ms is written on every entry and never read.

77:    recorded_at_ms: u64,          // field
4142:  recorded_at_ms: current_time_millis(),   // written
6136:  recorded_at_ms: 1,            // test fixture
6247:  assert!(entry.recorded_at_ms > 0);       // test assertion

No call site consumes it for pruning. The timestamp is carried for an eviction policy that isn't there.

The cost is the write pattern, not the bytes

71 KB is nothing. But write_artifact_cache_key_memo_file serializes and rewrites the entire map on every newly-seen root, so the per-new-root cost grows with the number of roots ever seen. That lands hardest on your own test suite: a run that configures hundreds of temp roots does hundreds of full-file rewrites of a monotonically growing file, and permanently pollutes the developer's storage dir while doing it.

Fix directions

Any one of these closes it:

  1. Drop entries whose root no longer exists, in the read path — one Path::exists() per entry, and the map is already fully deserialized there. Cheapest correct fix, and it self-heals existing files.
  2. Use recorded_at_ms — evict past some age. The field is already there.
  3. Don't let tests write to real storage — point the memo at the test's temp storage root. That fixes the 93% source rather than the symptom, and is probably worth doing regardless of 1 or 2.

I'd suggest 1 + 3: existence-pruning is trivially correct and self-healing, and test isolation stops the pollution at its source.

Not urgent — no user-visible symptom today, and I found it while auditing something else. Filing because it's a slow-growing registry with a timestamp field that reads like the eviction was meant to exist.

Verified against 83fe3bd5. The counts above are from a live 2-month-old storage dir.

Activity

  1. ualtinok commented on Aug 16, 2026

    @ualtinok
    Collaborator

    Fixed on main (lands in the next patch release), both halves:

    Eviction: cache-keys.json now prunes on every successful write — entries are removed when the root no longer exists and the entry is older than 7 days (both conditions, so a temporarily unmounted root or a long-idle live project survives). Read hits on existing roots refresh their timestamp at most once per day, so a root that is only ever read-borrowed stays retained. recorded_at_ms finally earns its keep. A seeded 400-dead-entry file shrinks on the next write; that's a regression test now.

    The 407 /tmp fixtures: those were ours — test contexts defaulted to no storage_dir, which resolves to the real data root, so integration runs leaked memo entries into your storage. Test contexts now use isolated temp storage, with a regression asserting an explicit storage_dir receives the memo and the default root stays clean.

    Good find — the "written but never read" observation on recorded_at_ms was exactly right.

  2. ualtinok commented on Aug 16, 2026

    @ualtinok
    Collaborator

    Shipped in v0.51.0 — both halves: cache-keys.json prunes on every successful write (entries removed when the root no longer exists and the entry is older than 7 days; read hits refresh at most daily so read-only roots stay retained), and the test-isolation leak that produced your 407 /tmp entries is fixed with a regression asserting the default storage root stays clean. recorded_at_ms is finally load-bearing. Good find on the written-but-never-read observation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions