Repository navigation
Events on before and after garbage collection (to aid service readiness reporting to load balancers) #58002
Description
What is the problem this feature will solve?
Our use case relates to garbage collection events (especially stop-the-world full sweeps) and how might we turn off the traffic from the service prior the garbage collection starts. Even in case of parallel GC, we would like to direct the traffic away in some cases, so that no requests would queue up faster than the service can process them. We are running node.js in Kubernetes pods with a load balancer in front. We have both stateful (as in websocket) and stateless (as in http) traffic, and with the stateless requests the payloads can be quite large (tens of megabytes), so the memory allocations are spiky rather than steady.
In our experience, while garbage collection is happening, the service can stop for more than a minute. Load balancer will notice this via readiness checks, but before the readiness check has completed, a few thousand requests may have already been lost and web sockets have stalled. We would like to minimise such outages.
If we could get an event a bit up-front that garbage collection is imminent, we could signal load balancers to direct the traffic away and disconnect web-sockets, so that all the traffic would go to other pods while garbage collection is running. When the garbage collection (full sweep) is complete, we could signal the load balancer that we are live again.
What is the feature you are proposing to solve the problem?
V8 module could have events on imminent (before) and after garbage collection has run (after). The event could carry information about the type of GC is going to be run.
What alternatives have you considered?
We already tried periodical checks of v8.getHeapSpaceStatistics() to identify memory pressure up-front, but this did not work; when we face memory pressure and stop the traffic, no new memory needs to be allocated and as a result GC might not run.
Activity
bnoordhuis commentedon Apr 26, 2025
You can do that from a native C++ add-on with
isolate->AddGCPrologueCallback()(edit: andisolate->AddGCEpilogueCallback()) - that's basically how every APM vendor works.What you want to do from JS land cannot possibly work, however, because the GC callback isn't allowed to allocate, and that prohibits it from calling into JS code.
If you have 1 minute GC stalls, there's something pathologically wrong with your program. I troubleshoot performance problems all the time (it's basically my bread and butter) and multiple second stalls are already rare, let alone anything that's a magnitude worse.
Are you sure it's not something else, like an event loop stall caused by a busy-looping callback?
ljharb commentedon Apr 26, 2025
You can use a FinalizationRegistry in JS to fire a callback after a target object is collected.
laurisvan commentedon Apr 27, 2025
@bnoordhuis Thanks, I'll check if this would be my first journey to the native addons (gasp).
The 1 minute GC stalls might be something specific to pointer compression branch we are using and/or Alpine linux musl malloc. We suspected memory fragmentation and tried jemalloc. But indeed, now that I check the logs, I only see a few seconds GC stall until OOM.
Regardless, I will close this one as I got the pointers I'm looking for - e.g. try the
isolate->AddGCPrologueCallback().