Repository navigation
RSS growing into several hundreds of megabytes and never going down in size in any significant way when no activity is being served. #56641
Description
Version
18.20.4
Platform
FreeBSD 14.1
Subsystem
No response
What steps will reproduce the bug?
I would like to ask if following behaviour of RSS part of memory of node process
is within a norm. Or is this an abnormal behaviour caused by memory leaks in some ibraries.
It's hard to judge based on valgrind, as it producess several dozens of thousands of lines of log
covering several libraries after termination of server.js process by Ctrl-C.
mkdir certs
openssl req -x509 -newkey rsa:2048 -nodes -sha256 -subj '/CN=localhost' -keyout certs/cert.key -out certs/cert.pem
node server.js
run several instances of at the same time
node client
RSS starts to grow into region of several hundred of megabytes when multiple instances of cliet.js are connecting to it.
No substantial decrease of size of RSS reported by server.js been observed
in time period of up to one hour. It decreased only by few dozens of megabytes
shortly after connections stopped but remained elevated into many hundreds
of megabytes region.
It's unclear if that memory would ever be available to the OS in case on need for memory by other processes. There are many environments with limited resources where node holding right to big areas of memory without actual need to hold it for prolonged periods of time could cause problems and starve other processes of needed memory. This is why some people could find it concerning.
After terminating (Ctrl+C) server.js running as result of
valgrind --leak-check=full --show-leak-kinds=all node server.js
Few dozens of thousands of lines is being generated covering many libraries.
Very last lines of them are following
==79512==
==79512== LEAK SUMMARY:
==79512== definitely lost: 0 bytes in 0 blocks
==79512== indirectly lost: 0 bytes in 0 blocks
==79512== possibly lost: 0 bytes in 0 blocks
==79512== still reachable: 5,348,193 bytes in 23,822 blocks
==79512== suppressed: 12,556 bytes in 37 blocks
==79512==
==79512== Use --track-origins=yes to see where uninitialised values come from
==79512== For lists of detected and suppressed errors, rerun with: -s
==79512== ERROR SUMMARY: 24 errors from 12 contexts (suppressed: 0 from 0)
At this point I'm not sure if adding full log would be of any help as it's szie
reached over 3 megabytes.
Files below:
server.js
const process = require('node:process');
const https = require('https');
const fs = require('fs');
//const heapdump = require('heapdump');
const port = ; //TODO set port number
process.on('uncaughtException', (err, origin) => {
console.log(err);
fs.writeSync(
process.stderr.fd,
`Caught exception: ${err}\n` +
`Exception origin: ${origin}\n`,
);
});
const privateKey = fs.readFileSync(__dirname + '/certs/cert.key');
const certificate = fs.readFileSync(__dirname + '/certs/cert.pem');
const options = { key: privateKey,
cert: certificate, enableTrace: false };
const server = https.createServer(options, (req, res) => {
res.write("Hello there!");
// res.writeHead(200);
res.end();
});
server.on('error', (e) => {
console.error(e);
});
server.on('connection', (socket) => {
//console.log('New connection established.');
socket.on('error', (err) => {
console.error('Socket error:', err);
socket.destroy(); // Close in case of error
});
socket.on('close', () => {
//console.log('Socket closed.');
});
});
server.listen(port, () => {
console.log(`Server listen on https://:${port}`);
});
setInterval(function(){
const formatMemoryUsage = (data) => `${Math.round(data / 1024 / 1024 * 100) / 100} MB`;
const memoryData = process.memoryUsage();
const memoryUsage = {
rss: `${formatMemoryUsage(memoryData.rss)} -> Resident Set Size - total memory allocated for the process execution`,
heapTotal: `${formatMemoryUsage(memoryData.heapTotal)} -> total size of the allocated heap`,
heapUsed: `${formatMemoryUsage(memoryData.heapUsed)} -> actual memory used during the execution`,
external: `${formatMemoryUsage(memoryData.external)} -> V8 external memory`,
};
console.log(memoryUsage);
}, 5*1000);
setInterval(function(){
server.getConnections((err, count) => console.log('Active connections: ' + count));
if(global.gc){
global.gc();
console.log("garbage free");
}
}, 5*1000);
/**
setTimeout(function () {
const filename = Date.now() + '.heapsnapshot';
heapdump.writeSnapshot(function(err, filename) {
// console.log('dump written to', filename);
});
}, 10*1000);
setTimeout(function () {
const filename = Date.now() + '.heapsnapshot';
heapdump.writeSnapshot(function(err, filename) {
// console.log('dump2 written to', filename);
});
}, 120*1000);
**/
client.js
const https = require('https');
const fs = require('fs');
const port = ; //TODO set port number here
const host = 'host address here'; //TODO set host name here
const requestCount = 3000;
const options = {
hostname: host,
port: port,
path: '/',
method: 'GET',
rejectUnauthorized: false
};
function makeRequest(id) {
const req = https.request(options, (res) => {
let data = '';
res.on('data', (chunk) => {
data += chunk;
});
res.on('end', () => {
console.log(`Request ${id} completed. Response: ${data}`);
});
});
req.on('error', (e) => {
console.error(`Request ${id} encountered an error: ${e.message}`);
console.error(e);
});
req.end();
}
for (let i = 1; i <= requestCount; i++) {
makeRequest(i);
}
How often does it reproduce? Is there a required condition?
Always.
What is the expected behavior? Why is that the expected behavior?
Expected behavior: Passing control over unused memory back to the OS.
Rationale: There are many environments with limited resources where node holding right to big areas of memory without actual need to hold it for prolonged periods of time could cause problems and starve other processes of needed memory. This is why some people could find it concerning.
What do you see instead?
RSS growing into hundreds of megabytes and never decreasing significantly even after very long time.
Additional information
No response
Activity
joyeecheung commentedon Jan 17, 2025
What memory allocator are you using? Have you tried using a memory allocator that prioritizes low RSS, like jemalloc? (Not sure if
LD_PRELOADworks on FreeBSD, though, so you might need to find another way to customize the memory allocator used). It's a known issue on Linux at least that if you use glibc's memory allocator, it can retain a huge amount of memory without giving it back to the OS even though it's freeable, and it could be a 1.5GB v.s. 70MB difference when one switches to jemalloc viaLD_PRELOADon Linux IIRC. In that case there's not much Node.js can do since it depends on the memory management library you use.reportingstuffandthings commentedon Jan 17, 2025
This is running on FreeBSD (amd64 compatible CPU).
It's libc's malloc() implementation is jemalloc (for many years now).
List of libraries generated by ldd executed on node binary
joyeecheung commentedon Jan 26, 2025
Are you able to reproduce it on other platforms or is it just FreeBSD?
reportingstuffandthings commentedon Jan 27, 2025
I have never tested it on any other platform.
bnoordhuis commentedon Feb 3, 2025
Is your question: is that normal? Yes.
JS is a GC language and V8's GC is memory hungry by default. There are several flags for enforcing size limits on the JS heap.
I'll go ahead and close this because AFAICT a) it's working as intended, and b) there's no indication of an actual memory leak.
reportingstuffandthings commentedon Feb 3, 2025
OK. Thank you for clearing that out.
alexey-sh commentedon May 16, 2025
bnoordhuis OP asked about RSS. V8 sits in the heap not in the RSS.
bnoordhuis commentedon May 16, 2025
@alexey-sh I don't understand how you arrived at the notion that "V8 sits in the heap not in the RSS" but it's wrong. Not wrong in the sense it lacks some nuance, wrong as in completely 100% missing the mark capital W wrong.
alexey-sh commentedon May 16, 2025
bnoordhuis It is possible to keep nodejs memory consumption(heap, v8) within 10 megabytes and see growing rss consumption up to 512 megabytes. V8 contains nodejs data, while nodejs handles sockets which is not part of the V8. So in case if you create a lot of sockets you can see that V8 memory consumption is low and RSS is very high.
This is common case and it does not related to the "V8's GC is memory hungry by default" at all by the nature.
bnoordhuis commentedon May 17, 2025
Yeah, no. What you describe a) doesn't happen1, and b) is irrelevant to the test case OP posted. Honestly, if you want to debate with the experts, you need to do better than vague hypotheticals. If you can't come up with cold, hard results, then just drop it already.
1 Even with TLS sockets in their most pathological configuration the overhead is fixed to something like 20kb per socket (but, in practice, much less) so that works out to 200mb for 10,000 sockets.
At that point your JS code is presumably doing interesting stuff with all that incoming data, so the JS heap is going to be a multiple of that because V8 implements the classic time/space trade-off where it prefers to grow the heap rather than vacuuming it more and more often.
"But data in buffers," you say? If you retain buffers longer than needed, sure, but that's an application issue, not a node bug.
alexey-sh commentedon May 18, 2025
Yeah, yes.
bnoordhuis commentedon May 18, 2025
Don't be annoying. If you believe you have a test case where node acts pathologically, open a new issue.