Skip to content

[plugin-multi-tenant] Race condition in auto-injected relationship field validation produces false "invalid relationship" errors #16462

Open

Description

@amitzur

Describe the Bug

When a tenant-enabled collection is referenced from an array field on another tenant-enabled collection, creating a document with multiple rows that each reference a tenant-scoped item intermittently fails with per-row "This relationship field has an invalid relationship" validation errors — even though every referenced id exists, belongs to the correct tenant, and is otherwise valid.

The failure is non-deterministic and the rate scales with row count. With 8 rows against MongoDB it reproduces in roughly 1–4 of every 5 attempts; with more rows it approaches 100%.

Suspected root cause

The multi-tenant plugin auto-injects filterOptions on every relationship pointing at a tenant-enabled collection. Payload's relationship validator then runs validateFilterOptions, which calls payload.find to check the chosen id is in the allowed set.

Payload validates array rows in parallel via Promise.all. With N rows each holding a tenant-scoped relationship, N concurrent payload.find calls run on the same MongoDB session inside the create's transaction. MongoDB rejects all but one with:

MongoServerError: Given transaction number X does not match any in-progress transactions on session Y

validateFilterOptions swallows the error in its catch and returns an empty allow-list, so the field is reported as "invalid" for every row whose query lost the race.

This is structurally adjacent to #4078 / #4087 — that fix made validateFilterOptions honor the transaction, but didn't address N validators in the same transaction trying to use the session concurrently. The multi-tenant plugin makes this very easy to hit because it auto-injects filterOptions on every tenant-scoped relationship — the user's schema never declares one.

Link to the code that reproduces this issue

https://github.com/amitzur/payload-issue-multi-tenant-relationship-validation-race

Reproduction Steps

Run the reproducer script from the repo amitzur/payload-issue-multi-tenant-relationship-validation-race:

pnpm install
pnpm repro

Expected: the test passes — failures stays empty across all 5 trials.
Actual: the test fails — one or more trials produce per-row "This relationship field has an invalid relationship" errors.

The reproducer registers three collections via the multi-tenant plugin:

  • tenants
  • items (tenant-enabled)
  • holders (tenant-enabled, has rows: array of relationship → items)

It then creates one tenant + 8 items in that tenant, then attempts five times to create a holder whose rows reference all 8 items. The expectation is that all five succeed; in practice one or more fail with per-row validation errors.

Race condition proof

Uncomment the following to make the repro pass: https://github.com/amitzur/payload-issue-multi-tenant-relationship-validation-race/blob/62ba2a89245934f7d9ec59f4df1a42db377af8e1/payload.config.ts#L74

Which area(s) are affected?

db: mongodb, plugin: multi-tenant

Environment Info

Node.js: v24.12.0
payload: 3.84.1
@payloadcms/plugin-multi-tenant: 3.84.1
@payloadcms/db-mongodb: 3.84.1
Next.js: 16.2.4

Activity

  1. added
    status: needs-triagePossible bug which hasn't been reproduced yet
    validate-reproductionAuto-added tag on create to tell bot to check recreation URL, removed after check.
    on May 4, 2026
  2. added
    db: mongodb@payloadcms/db-mongodb
    and removed
    validate-reproductionAuto-added tag on create to tell bot to check recreation URL, removed after check.
    on May 4, 2026
  3. remiroyc commented on Sep 30, 2026

    @remiroyc

    We hit this in production without plugin-multi-tenant, so the race looks like a core / db-mongodb issue rather than a plugin one.

    Setup

    • payload 3.88.0, @payloadcms/db-mongodb 3.88.0, MongoDB Atlas (replica set)
    • A collection with a blocks field. Several blocks contain a plain upload field with a static filterOptions (a mime type filter):
    {
      name: 'video',
      type: 'upload',
      relationTo: 'blog-media',
      filterOptions: { mimeType: { in: ['video/mp4', 'video/quicktime'] } },
    }

    What happens

    Creating a document with two or more of these fields and publishing it in the same request (POST /api/posts, no draft) intermittently fails with The following fields are invalid: … Video. The referenced uploads exist and match the filter. The server logs one Error validating filter options for collection blog-media per failed field, with:

    MongoServerError: Given transaction number 1 does not match any in-progress transactions. The active transaction number is -1
    

    (NoSuchTransaction, code 251), and on later attempts Transaction … has been aborted.

    Only create is affected. Saving as a draft first and then publishing always works, which fits the analysis above: updateByID reads the existing document on the session before validating, so the transaction already exists on the server when the concurrent validateFilterOptions finds start. On create, those finds are the first commands on the session.

    Reproduction

    With our config against a local single-node replica set, creating a document with 6 such upload blocks and draft: false failed in 6 of 15 runs with the same error.

    Workaround

    A collection beforeOperation hook that awaits one read on the request's session before validation. With it, 0 of 8 runs failed in the same repro:

    const openTransactionBeforeCreate: CollectionBeforeOperationHook = async ({ args, collection, operation, req }) => {
      if (operation !== 'create' || !req.transactionID) return args
      await req.payload.db.findOne({ collection: collection.slug, req, select: { id: true }, where: {} })
      return args
    }

    This only makes sure the transaction is started before the parallel finds; the finds still run concurrently on the session afterwards. Serializing them as in #16463 looks like the proper fix, and it would cover our case too since it isn't specific to the multi-tenant plugin.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions