Skip to content

Bulk assignment delete makes one gateway call and one database delete per assignment, serially, with no cap on the batch #1749

Open

Description

@joshunrau

POST /v1/assignments/bulk/delete removes assignments strictly one at a time. For each outstanding assignment it makes a separate HTTP DELETE to the gateway, then a separate MongoDB delete, and waits for both before starting the next. The gateway client is configured with keepAlive: false (gateway.http.ts), so every one of those calls also opens a fresh TCP connection, plus a TLS handshake when the gateway is reached through its HTTPS site address. $DeleteBulkAssignmentsData.ids has no upper bound. The Delete Assignments page has a select-all toggle over every outstanding assignment in the group, and a single bulk create can produce 500 subjects × M timepoints of them. Creation already sends the whole batch to the gateway in one request (createRemoteAssignments → POST /api/assignments/bulk); deletion has no such path. The web sends the request with the default 10-second timeout. Once a group has a few hundred outstanding assignments, the browser therefore gives up with "Network Error" while the API keeps deleting in the background. The user sees a failure for a delete that is partly or wholly done, and retrying sends the same long serial request again.

Where

apps/api/src/assignments/assignments.service.ts:158-172:

for (const assignment of assignments) {
  try {
    if (assignment.status === 'OUTSTANDING') {
      await this.gatewayService.deleteRemoteAssignment(assignment.id);
    }
    await this.assignmentModel.delete({ where: { id: assignment.id } });
    deletedIds.push(assignment.id);
  } catch (err) {

packages/schemas/src/assignment/assignment.ts:180-182: no cap on the batch:

const $DeleteBulkAssignmentsData = z.object({
  ids: z.array(z.string().min(1)).min(1)
});

apps/api/src/gateway/gateway.http.ts:29-30: a new connection per call:

httpAgent: new http.Agent({ keepAlive: false }),
httpsAgent: new https.Agent({ keepAlive: false })

Reproduce

Mechanism: bulk-assign one instrument to 500 subjects, open Group → Remote Assignments → Delete Assignments, select all, and delete. The API makes 500 sequential gateway round trips, each with its own connection setup and SQLite delete, and 500 sequential MongoDB deletes. At a realistic 20–50 ms per gateway call that is 10–25 s, past the web client's 10 s timeout (apps/web/src/services/axios.ts; useDeleteBulkAssignmentsMutation does not set disableDefaultTimeout). The work grows linearly with the selection and nothing bounds the selection.

Tests

  • apps/api/src/assignments/__tests__/assignments.service.test.ts: it('should remove a batch from the gateway in one call and from the database in one deleteMany, rather than once per assignment').
  • apps/gateway/src/routers/__tests__/api.router.test.ts: a bulk delete route that removes every listed id in one transaction and reports the ids it did not find.
  • packages/schemas: $DeleteBulkAssignmentsData refuses more ids than the documented cap.
  • testing/src/specs/remote-assignments-wizard.spec.ts: delete a multi-subject batch with select-all and assert that every row is gone and a success message is shown.

Suggested fix

Add a gateway POST /api/assignments/bulk/delete that deletes a list of ids in one transaction, mirroring the bulk create. Call it once from deleteBulk with the outstanding ids, then remove the deleted rows with a single assignmentModel.deleteMany, and report the ids the gateway could not remove as failedIds. Cap ids at a bound consistent with creation, for example BULK_ASSIGNMENT_MAX_SUBJECTS × timepoints, or a fixed maximum the web chunks against.

Activity

  1. added
    EnhancementType: an improvement to an existing feature, including performance
    Difficulty: MediumStraightforward change, about a day or less
    Area: APIapps/api (NestJS backend, Prisma models, CASL permissions)
    Area: Gatewayapps/gateway (patient-facing app for remote assignments)
    PerformanceSpeed, memory or scalability
    on Oct 7, 2026
  2. joshunrau commented on Oct 26, 2026

    @joshunrau
    CollaboratorAuthor

    Still applies on main (d86740e). Nothing has touched deleteBulk, the gateway router or $DeleteBulkAssignmentsData since this was filed. No e2e spec exercises POST /v1/assignments/bulk/delete or ?mode=delete yet.

    An id the gateway does not hold must count as removed

    The suggested fix reports "the ids the gateway could not remove" as failedIds. That must not include ids the gateway never had. Today an outstanding assignment with no gateway copy can never be deleted. DELETE /api/assignments/:id answers 404 (apps/gateway/src/routers/api.router.ts:165-167). gateway.http.ts sets no validateStatus, so axios rejects the 404 and the id lands in failedIds on every retry. Cancel (assignments.service.ts:203-205) fails on the same 404.

    A row ends up like that when:

    • the gateway delete in deleteBulk (:161) succeeds and the Mongo delete (:163) throws;
    • handleAssignmentComplete deletes the gateway copy (gateway.synchronizer.ts:216) and the updateStatusById at :94 then fails. The row stays OUTSTANDING, and the gateway no longer lists it, so no sync pass retries it;
    • the gateway's SQLite database is lost or recreated.

    Counting a missing id as removed is also what lets a retry converge after a partial failure.

    Keep rows the participant already submitted

    The gateway deletes a row even when it has been submitted (completedAt is set at api.router.ts:142). Until the next sync pass imports it (GATEWAY_REFRESH_INTERVAL, 10 s in .env.template), the API row is still OUTSTANDING, so deleting it loses the submission. Cancel has the same gap. The new route can close it without an interactive transaction: deleteMany({ where: { id: { in: ids }, completedAt: null } }), then read back which listed ids remain. Those are the submitted ones. Return them, and treat every other id as removed. The API reports them in failedIds and leaves them for the synchronizer to import.

    Use a fixed cap and chunk in the web

    BULK_ASSIGNMENT_MAX_SUBJECTS × timepoints is not a constant, because timepoints has no .max() (packages/schemas/src/assignment/assignment.ts:80-85). The delete page also lists every outstanding or expired assignment in the group, however it was created (DeleteRemoteAssignments.tsx:54-64). Select-all can therefore exceed any bound tied to creation. Instead:

    • Export a fixed BULK_ASSIGNMENT_DELETE_MAX_IDS (500) and validate ids: $UniqueStrings.max(BULK_ASSIGNMENT_DELETE_MAX_IDS), as subjectIds already does at :79. Apply the same bound to the gateway body.
    • Have useDeleteBulkAssignmentsMutation send sequential chunks and merge { deletedCount, failedIds }, so Bulk-deleting remote assignments reports success when the server could not delete them #1685 reads one result. Move the query invalidation from onSuccess to onSettled: a chunk that throws leaves the earlier chunks already deleted.

    Smaller points

    Tests

    In addition to the ones in the issue:

    • packages/schemas/src/assignment/__tests__/assignment.test.ts: refuses duplicate ids.
    • apps/gateway/src/routers/__tests__/api.router.test.ts: keeps a submitted row and returns its id. An unknown id is not an error.
    • apps/api/src/gateway/__tests__/gateway.service.test.ts: the bulk delete makes one POST.
    • apps/api/src/assignments/__tests__/assignments.service.test.ts: four of the six deleteBulk cases (:452-510) mock one gateway call and one delete per id and need rewriting. Add one for a missing gateway copy and one for a kept submitted row.
    • apps/web/src/hooks/__tests__/useDeleteBulkAssignmentsMutation.test.ts: splits a selection above the cap and merges failedIds.
    • e2e: testing/src/specs/bulk-remote-assignments.spec.ts already drives the real gateway through the API. Create a 2×2 batch, delete all four, and expect { deletedCount: 4, failedIds: [] } and a 404 from each assignment.url. Also expect a 400 above the cap. The select-all UI case belongs in remote-assignments-wizard.spec.ts. PR fix(web): align remote assignment table colors with datahub subject table #1811 and PR fix: resolve 26 low-difficulty issues, one commit each #1825 both edit that spec, and fix(web): align remote assignment table colors with datahub subject table #1811 also edits DeleteRemoteAssignments.tsx, so rebase on both.

    Related

    Triage note written with Claude Code. Code references point at main @ d86740e.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Area: APIapps/api (NestJS backend, Prisma models, CASL permissions)Area: Gatewayapps/gateway (patient-facing app for remote assignments)Difficulty: MediumStraightforward change, about a day or lessEnhancementType: an improvement to an existing feature, including performancePerformanceSpeed, memory or scalabilityPriority: MediumReal value; schedule normally

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions