Repository navigation
buffer: optimize Buffer.toString('hex') for small buffers with stack allocation #194
Description
While looking into buffer optimizations (nodejs/node#61871), I noticed that Buffer.toString('hex') heap allocates a temporary buffer for every call even for tiny inputs like a SHA-256 digest
I tried using a 512 byte stack buffer for small inputs (<=256 bytes / 512 hex chars) and calling NewFromOneByte directly, skipping ExternOneByteString entirely. This covers all the common cases SHA-256, SHA-512, UUIDs, HMAC signatures, MongoDB ObjectIds, etc.
Let me know what you guys think of this (runs=60)
confidence improvement accuracy (*) (**) (***)
buffers/buffer-tostring.js n=1000000 len=1 args=1 encoding='hex' *** 34.44 % ±1.05% ±1.39% ±1.80%
buffers/buffer-tostring.js n=1000000 len=1 args=3 encoding='hex' *** 32.38 % ±2.10% ±2.79% ±3.61%
buffers/buffer-tostring.js n=1000000 len=1024 args=1 encoding='hex' *** -1.11 % ±0.62% ±0.82% ±1.07%
buffers/buffer-tostring.js n=1000000 len=1024 args=3 encoding='hex' -0.84 % ±0.88% ±1.17% ±1.51%
buffers/buffer-tostring.js n=1000000 len=128 args=1 encoding='hex' *** 29.00 % ±1.67% ±2.22% ±2.88%
buffers/buffer-tostring.js n=1000000 len=128 args=3 encoding='hex' *** 30.72 % ±0.79% ±1.05% ±1.35%
buffers/buffer-tostring.js n=1000000 len=256 args=1 encoding='hex' *** 26.65 % ±0.93% ±1.23% ±1.59%
buffers/buffer-tostring.js n=1000000 len=256 args=3 encoding='hex' *** 27.13 % ±1.46% ±1.93% ±2.49%
buffers/buffer-tostring.js n=1000000 len=512 args=1 encoding='hex' ** -1.24 % ±0.85% ±1.12% ±1.44%
buffers/buffer-tostring.js n=1000000 len=512 args=3 encoding='hex' ** -0.63 % ±0.41% ±0.55% ±0.71%
buffers/buffer-tostring.js n=1000000 len=64 args=1 encoding='hex' *** 37.50 % ±1.65% ±2.18% ±2.81%
buffers/buffer-tostring.js n=1000000 len=64 args=3 encoding='hex' *** 39.28 % ±1.29% ±1.72% ±2.23%
Be aware that when doing many comparisons the risk of a false-positive
result increases. In this case, there are 12 comparisons, you can thus
expect the following amount of false-positive results:
0.60 false positives, when considering a 5% risk acceptance (*, **, ***),
0.12 false positives, when considering a 1% risk acceptance (**, ***),
0.01 false positives, when considering a 0.1% risk acceptance (***)
Activity
lemire commentedon Mar 5, 2026
Do you have the diff ? (So we can see your code?)
thisalihassan commentedon Mar 5, 2026
Sure here's the diff @lemire
thisalihassan commentedon Mar 7, 2026
quick follow up @nodejs/performance any thoughts on this approach? Happy to open a PR if this looks reasonable!
ChALkeR commentedon Mar 7, 2026
yes, the whole string_bytes is pretty unoptimal for non-externalized strings in a bunch of codepaths, not just hex
i wonder if it can be optimized in a more generic way
thisalihassan commentedon Mar 8, 2026
yeah totally @ChALkeR I have been looking into this more. The pattern is the same across all encodings every encode call does a malloc, encodes into it, V8 copies from it, then frees it, even for tiny strings. Here's the idea, for small inputs (<=512 chars) just encode into a stack buffer and let V8 copy directly from that, for large strings keep the existing external string path. Wrapped it in a generic helper so hex, base64, base64url, ascii, and the two-byte paths (utf8, ucs2) all benefit from the same thing
src: use stack allocation for small string encoding
src: use stack allocation for small string encoding
src: use stack allocation for small string encoding
src: use stack allocation for small string encoding