API and gateway runtime images ship the full Node build image with gcc, git and turbo, tripling their size #1794

Activity

1. joshunrau

   added

   Priority: LowNice to have, cosmetic or speculativeNice to have, cosmetic or speculative

   EnhancementType: an improvement to an existing feature, including performanceType: an improvement to an existing feature, including performance

   Difficulty: MediumStraightforward change, about a day or lessStraightforward change, about a day or less

   Area: InfrastructureDeployment, Docker, CI, logging, CLIs, build and test toolingDeployment, Docker, CI, logging, CLIs, build and test tooling

   PerformanceSpeed, memory or scalabilitySpeed, memory or scalability

   SecurityAuthentication, authorization, data exposure or a vulnerable dependencyAuthentication, authorization, data exposure or a vulnerable dependency

   on Oct 7, 2026
2. joshunrau commented on Oct 26, 2026

   

   joshunrau

   on Oct 26, 2026

   CollaboratorAuthor

   More actions

   Still reproduces on main (d86740e). Both runners are still FROM base (apps/api/Dockerfile:25, apps/gateway/Dockerfile:27), and no commit since this was filed touches either file.

   Corrections to the suggested fix

   • Keep ENV NODE_OPTIONS="--max-old-space-size=8192" in the api runner. If it moves to the build stages only, the API falls back to V8's default heap limit. That default is about 2 GiB on a host with less than 15 GiB of memory and about 4 GiB above that (Node 24.15 reports heap_size_limit as 4288 MiB on an 18 GiB host). Performance: /instrument-records/export buffers the whole dataset in memory (12.8s, +2.9 GiB RSS, 122 MiB body at 200k records) #1407 measured 3.5 GiB of heap for one 200k-record export at 30 measures per record, so that export would run out of heap on any host under 15 GiB. Keep the flag in the gateway runner too, so this PR changes only the packaging. Whether 8 GiB is the right limit belongs in Performance: /instrument-records/export buffers the whole dataset in memory (12.8s, +2.9 GiB RSS, 122 MiB body at 200k records) #1407.
   • The new runners lose two things base provides today:
     • WORKDIR /app. The slim image sets none, so the RUN echo … > package.json lines and the relative CMD paths would resolve from /.
     • In the gateway, ENV GATEWAY_DATABASE_URL=file:/app/sqlite/gateway.db (apps/gateway/Dockerfile:4). Compose does not pass it (docker-compose.yaml:82-84). run.sh expands it under set -eu (apps/gateway/scripts/run.sh:3,5), so without it the container exits at start. Prisma reads it too (apps/gateway/prisma/schema.prisma:12). Keep it in base as well, because the installer's db:push (apps/gateway/Dockerfile:24) creates the template database at that path.
   • Tag: node:24-slim and node:krypton-slim are the same bookworm-slim image, the same Debian release as lts-krypton. There is no lts-krypton-slim. Avoid lts-slim: it moves to Node 26 when that line enters LTS on 2026-10-28 (tags, schedule).
   • Do not go on to alpine. The build stage is glibc, so the native files the runners load are glibc builds. The Prisma engine is picked at build time by getBinaryTargetForCurrentPlatform, in libnest's src/meta/plugins/prisma.ts for the api and in @douglasneuroinformatics/esbuild-plugin-prisma for the gateway. The libstats addon in the api's dist/ is the libc: [glibc] variant, the only one pnpm installs there.

   Verification to add

   • Run ldd /app/dist/*.node in the api image and ldd /.prisma/client/*.node in the gateway image, on both architectures. A not found line is a missing shared library, and the check takes seconds without the full stack.
   • The issue's checklist does not run the bundled esbuild binary or the export worker. Also create a series instrument (POST /v1/instruments/series calls bundle() at apps/api/src/instruments/instruments.service.ts:198), and export from the datahub, which starts export-worker.js and resolves #runtime through the runner's generated package.json.
   • No CI job builds these images on a PR. On main, should_release is always 'true' (.github/scripts/release.cjs:25), so every push builds both images for linux/amd64 and linux/arm64 (release.yaml:89) and pushes them as latest (:79). The merge is the first CI build, and it goes straight to every deployment that pulls latest. Build and check both architectures locally before merging. Once ci: release from release branches and keep latest on tagged releases #1822 lands, a push to main publishes edge instead, which gives a target to smoke-test before latest moves.
   • Root AGENTS.md:49-51 exempts only scripts/ from the unit and e2e rule. State this exception in the PR body, as ci: release from release branches and keep latest on tagged releases #1822 does for its workflow change.

   Landing order

   • Land this before Gateway container runs its internet-facing server as root #1792 or in the same PR, since both rewrite the gateway runner. Gateway container runs its internet-facing server as root #1792's setpriv approach still works on slim: /usr/bin/setpriv is in util-linux, which is Essential in bookworm.
   • ci: release from release branches and keep latest on tagged releases #1822 documents the Node pin as one FROM node:lts-krypton line per Dockerfile: the "Base images" section of its new release-channels.md, and step 2 of "From a release branch" in cut-a-release.md. After this change the api and gateway Dockerfiles each have a second FROM for the runner, and that is the line that ships Node to production. Whichever of the two PRs lands second should extend those docs so a release branch pins the runner too, for example to node:24.22.1-bookworm-slim.

   _{Triage note written with Claude Code. Code references point at main @ d86740e.}