Skip to content

Ordinary updates leak continuation chunks: 243821 orphans out of 1545495 chunk slots in CRUDTest.multiUpdatesOverlap #6319

Description

@lvca

Found by the orphaned-chunk sweep #6294 added (PR #6299), which is the first thing in the engine that can see this.

What happens

An ordinary update workload leaks continuation chunks. CRUDTest.multiUpdatesOverlap - 131072 records, each
given ten short string fields over thirteen rounds of updateAll - ends with:

chunk slots in the bucket : 1545495
reachable from any chain  : 1301674
reachable from nothing    :  243821     (16%)

Every one of those 243821 slots holds up to a page of content that no record can reach and nothing ever
frees. Nothing in the test is corrupt: no CHECK DATABASE error, no broken chain, no force-delete. It is the
plain update path.

This is not a mis-measurement

The sweep marks a chunk by walking the chain of every head, so "unmarked" could in principle mean "the walk
missed it". It was checked directly rather than assumed: for unmarked chunks, every slot of the bucket was
scanned for one whose next-chunk pointer names it.

  • #1:604567 - zero referrers of any kind.
  • #1:1095959 - one referrer, #1:727045, which is itself a NEXT_CHUNK and itself unmarked.

The second shape is the giveaway: these are orphaned sub-chains, chunk1 -> chunk2 -> chunk3 still pointing
at each other with nothing pointing at chunk1. A walk that had merely stopped early could not produce it,
because the walk marks a pointer as it reads it - reaching #1:727045 would have marked #1:1095959 too.

Where to look

updateMultiPageRecord rewrites a record's chain in place: chunks that are still needed are reused, chunks
that are not are handed to freeChunkChain. #6294's issue text already names the suspects - "an interrupted
updateMultiPageRecord shrink" and GraphEngine.rebuildVertexEdgeChain - as paths that "leave chunks behind
without deleting anything". The fixture here is a pathological one for chunking (a bucket where pages are
packed, so a spilled record's head chunk gets a handful of bytes and the chain runs to ~12 chunks for a
~700-byte record), which is presumably why it shows up so starkly.

The interesting question is which rewrite drops the reference: a chain that shortens, one that lengthens, one
that collapses, or the re-spill after a placeholder content record is deleted and recreated.

Why it is worth fixing at the source

CHECK DATABASE FIX now reclaims these, but that is a repair, not a cure:

A reproduction already exists in the tree and needs no fabrication: run CRUDTest.multiUpdatesOverlap and
read orphanedChunks from check database.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions