Skip to content

Reading a multi-page record fails with a ConcurrentModificationException when an unrelated record's chunk on the same page is written #6217

Description

@lvca

LocalBucket.readMultiPageRecord assembles a chunked record and then re-validates the page version of every page
the chain touched:

for (final long[] pv : pageVersions) {
  final BasePage currentPage = database.getPageManager().getImmutablePage(...);
  if (currentPage != null && currentPage.getVersion() != pv[2]) {
    chainInconsistent = true;
    break;
  }
}

A page version changes when any record on that page changes, and continuation chunks of different records routinely
share a page - the allocator packs them there on purpose. So a reader is failed by writes that did not touch a single
byte of its own record. After TX_RETRIES attempts (default 3) it gives up with

Multi-page record #1:26 was modified during read after N retries. Please retry the operation

This is the same false conflict the disjoint-slot merge removed from the WRITE path in #5381/#6129/#6175, still
present on the read path.

Measured

Issue6129ChunkedSlotMergeTest's own fixture is enough: 40 records whose head chunks live on page 0 and whose
continuation chunks all land on page 1 (verified by instrumenting the allocator - every one of the 40 placements is
headPage=0 avoid=0 -> page=1), then 8 threads rewriting their own records, 320 transactions.

With the read budget lowered to zero (database.getConfiguration().setValue(TX_RETRIES, 0)), the run produces
exactly this conflict and nothing else:

SCRATCH-READ conflicts=1 ofWhichReadSide=1
SCRATCH-READ-MSG Multi-page record #1:26 was modified during read after 0 retries. Please retry the operation

1 in 2 repetitions, and never a commit-side conflict: with the default budget of 3 the same race needs four
consecutive losses, which is what a loaded machine produces every few runs.

Why it matters beyond the test

  • A read of a multi-page record can fail on a busy bucket although the record is untouched, and the failure
    reaches the application as a ConcurrentModificationException - retryable, but a retry storm on a hot page is the
    cost, and the more records share a continuation page the likelier it is.
  • The commit side already answers this exact question precisely rather than by page version: #6129 introduced
    offPageContentFingerprint, a fold of the chain past the head chunk. A read could validate what it actually
    consumed - the chunk headers and content it followed - instead of the version of every page they happen to live on.

Where it surfaced

Issue6129ChunkedSlotMergeTest.growingUpdatesOfChunkedRecordsSharingAPageNeverConflict fails on a loaded machine
with expected 0 but was 2 (seen once in 4 full engine lane runs; passes in isolation and when its package runs
alone). The conflicts it counted were these read-side revalidations, not merge failures - the test now separates the
two and reports the counters, so the next failure says which mechanism produced it rather than only how many.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions