Found while building the guard for #6178, and confirmed on unmodified main (50eb961) - nothing in #6141 /
#6175 / #6178 causes it.
What happens
A SELECT over a type whose bucket holds a placeholder-backed record returns that record's content twice,
under two different RIDs, when the placeholder's CONTENT record was large enough that no page could host it whole.
select count(@rid) as c from Surrogate where v = <the 200 KB value> --> 2
Reproduced by Issue6178ChunkCollapseTest.aPlaceholderContentStoredAsAChainIsNeverCollapsed, which asserts the 2
deliberately (with a pointer here) so that fixing this turns that line red instead of leaving it quietly wrong.
Why
A content record is normally recognised by the NEGATIVE size marker createRecordInternal(record, isPlaceHolder=true, ...) writes, and every reader that walks a page skips it on that basis - LocalBucket.scan
returns a slot only for recordSize[0] > 0, RECORD_PLACEHOLDER_POINTER or FIRST_CHUNK.
But when the content record does not fit a page, that same method spills it through writeMultiPageRecord, which
writes FIRST_CHUNK - the marker of an ordinary multi-page record. The "this slot is somebody's content, not a
record" information is lost at that moment, and there is nowhere else it is written down: the head chunk of a
placeholder's content is indistinguishable from the head chunk of a real record.
So scan() hands it out as a document of its own, in addition to handing out the same bytes through the
placeholder POINTER that references it. check() has the mirror of the same confusion: it counts such a record
under totalMultiPageRecords and never under totalSurrogateRecords.
How to reach it
Both halves are needed: a page with a free tail of exactly zero (since #6149 the only shape that still produces a
placeholder pointer - see BucketPageLayoutTestSupport.sealFirstPage) and a content record bigger than a page.
sealFirstPage plus a 200 KB value does it. Databases written before #6149 contain placeholders far more freely,
so the population is larger there.
Worth considering
The marker namespace has room: a FIRST_CHUNK_PLACEHOLDER_CONTENT marker (the chunk head of a content record)
next to FIRST_CHUNK, written by writeMultiPageRecord when isPlaceHolder, and read wherever FIRST_CHUNK is
read today - scan, check, getRecordInternal, deleteRecordInternal, the free-space accounting, and the
disjoint-slot merge's SLOT_KIND_FIRST_CHUNK. That is a stored-format addition, so existing databases keep
producing the ambiguous shape and the readers have to tolerate both for ever.
The cheaper alternative is to stop creating the shape: refuse to place a content record that cannot fit a page,
i.e. make the spill branch of updateRecordInternal fall through to chunks for the RECORD ITSELF rather than
building a placeholder whose content then has to spill anyway. #6149 already removed every other reason to build
one; the remaining fallback exists because the slot cannot host a 14-byte chunk header, which says nothing about
whether the content will fit a page. That leaves nothing new in the format, and the readers keep tolerating what
old databases contain.
Found while building the guard for #6178, and confirmed on unmodified
main(50eb961) - nothing in #6141 /#6175 / #6178 causes it.
What happens
A
SELECTover a type whose bucket holds a placeholder-backed record returns that record's content twice,under two different RIDs, when the placeholder's CONTENT record was large enough that no page could host it whole.
Reproduced by
Issue6178ChunkCollapseTest.aPlaceholderContentStoredAsAChainIsNeverCollapsed, which asserts the 2deliberately (with a pointer here) so that fixing this turns that line red instead of leaving it quietly wrong.
Why
A content record is normally recognised by the NEGATIVE size marker
createRecordInternal(record, isPlaceHolder=true, ...)writes, and every reader that walks a page skips it on that basis -LocalBucket.scanreturns a slot only for
recordSize[0] > 0,RECORD_PLACEHOLDER_POINTERorFIRST_CHUNK.But when the content record does not fit a page, that same method spills it through
writeMultiPageRecord, whichwrites
FIRST_CHUNK- the marker of an ordinary multi-page record. The "this slot is somebody's content, not arecord" information is lost at that moment, and there is nowhere else it is written down: the head chunk of a
placeholder's content is indistinguishable from the head chunk of a real record.
So
scan()hands it out as a document of its own, in addition to handing out the same bytes through theplaceholder POINTER that references it.
check()has the mirror of the same confusion: it counts such a recordunder
totalMultiPageRecordsand never undertotalSurrogateRecords.How to reach it
Both halves are needed: a page with a free tail of exactly zero (since #6149 the only shape that still produces a
placeholder pointer - see
BucketPageLayoutTestSupport.sealFirstPage) and a content record bigger than a page.sealFirstPageplus a 200 KB value does it. Databases written before #6149 contain placeholders far more freely,so the population is larger there.
Worth considering
The marker namespace has room: a
FIRST_CHUNK_PLACEHOLDER_CONTENTmarker (the chunk head of a content record)next to
FIRST_CHUNK, written bywriteMultiPageRecordwhenisPlaceHolder, and read whereverFIRST_CHUNKisread today -
scan,check,getRecordInternal,deleteRecordInternal, the free-space accounting, and thedisjoint-slot merge's
SLOT_KIND_FIRST_CHUNK. That is a stored-format addition, so existing databases keepproducing the ambiguous shape and the readers have to tolerate both for ever.
The cheaper alternative is to stop creating the shape: refuse to place a content record that cannot fit a page,
i.e. make the spill branch of
updateRecordInternalfall through to chunks for the RECORD ITSELF rather thanbuilding a placeholder whose content then has to spill anyway. #6149 already removed every other reason to build
one; the remaining fallback exists because the slot cannot host a 14-byte chunk header, which says nothing about
whether the content will fit a page. That leaves nothing new in the format, and the readers keep tolerating what
old databases contain.