Http2 x http1 benchmark #5084
Replies: 9 comments 5 replies
|
can you provide a repo with the benchmarks you are attempting? |
|
I'll prepare one this weekend! But It consists just in the provided js file with latest undici and axios version installed. I'm running it on latest NodeJs 24, too. Also, I took a look at both, axios and undici http2 conn management. Axios does not use a connection pool, at least at first glance. That's why I've setup undici to 1 connection, so it wouldn't pool conns, too. That was the best setup I could figure out for this bench. Finally, I've ran it in inside wsl. I'll run it in a pipeline after I create the repo you asked to see if my local machine affects the bench anyhow |
|
@metcoder95 I've created the repository as requested. Resutls keeps consistent between node versions: undici beat axios by a large difference when using http1, but http2 axios is consistently faster https://github.com/Farenheith/undici-benchmark/actions/runs/24944781831/job/73044257456 |
|
Thanks for putting together the repro repo — I took a closer look and one thing stands out immediately on the Undici side:
get: (url) => {
const controller = new AbortController()
return undici.fetch(url, {
method: 'GET',
dispatcher: undiciAgentH2,
headers: new Headers({
Connection: 'keep-alive'
}),
signal: controller.signal
}).then((result) => {
if (!result.ok) throw new Error(`Request failed with status ${result.status}`)
return result
}).then((res) => res.text())
}Axios is not doing the equivalent in this benchmark. The repo just calls I tried a reduced harness with the same basic H2 workload, but changing the Undici calls as follows: Current repo-style Undici fetchawait undici.fetch(url, {
dispatcher: agent,
headers: new Headers({ Connection: 'keep-alive' }),
signal: new AbortController().signal
})Minimal Undici fetchawait undici.fetch(url, {
dispatcher: agent
})Undici requestawait undici.request(url, {
dispatcher: agent
})In my runs, the results looked like this:
So just removing the extra per-request setup on the That’s why I’d be careful about reading the current repo result as:
What it seems to show more specifically is:
One especially important point is that the Also, I checked connection usage on the H2 side: both clients were using 1 HTTP/2 session with 300 streams, so this does not appear to be a connection-count difference. The asymmetry in per-request setup looks like a more plausible explanation for at least part of the delta. I think a useful next step would be to benchmark these three side by side in the repo itself:
That would help separate fetch-layer overhead from transport performance. |
|
@mcollina, thank you for your feedback about my benchmark. AbortController was used because axios offers timeout control in its external api, so I wanted to make a fair comparison between both as handling request timeout is essential in most cases, but indeed I was using the wrong tool for the job; I should've used AbortSignal directly. Fixed it. Keep-alive headers are also pointless when using http2, so I removed it, too, from all h2 cases. I've also improved the benchmark, adding the cases you suggested. Here are the new results: https://github.com/Farenheith/undici-benchmark/actions/runs/24960277630/job/73085633502 What bugged me a little bit is that request + AbortSignal is consistently better than pure request, which doesn't make any sense, as minimal fetch is faster than axios, and fetch + AbortSignal is slower. So, I've changed another thing: I've removed The results are still the same. https://github.com/Farenheith/undici-benchmark/actions/runs/24960556515/job/73086342693 So, somehow AbortSignal has some impact in fetch, but in request, it makes it a bit faster? I know the difference is marginal, but I've rerun this many times locally, and the rank doesn't change. |
|
I cloned Farenheith/undici-benchmark and ran it as-is, three times (median of 40 runs each): Node v24.14.1, axios 1.15.2, Windows; undici 8.1.0 (lockfile) for the first two runs and 8.10.0 for the third. I could not reproduce "axios beats undici on HTTP/2" — quite the opposite: Two things stand out:
Caveats on my runs: the HTTP/1.1 legs were noisy on Windows (300 concurrent sockets hit If you want a like-for-like h2 measurement, I'd suggest adding one more client to the matrix: |
|
Hi @Farenheith — thanks for the reply! Yes, my runs were on native Windows (Node v24.14.1, Git Bash). Appreciate you re-running on your Ubuntu box; the h2 ordering held across undici 8.1.0 and 8.10.0 on my side, so it will be interesting to see if WSL changes the h1 picture too. And yes — PR sent: Farenheith/undici-benchmark#1. It adds request failure counting alongside the medians (the h1 ECONNREFUSED bursts made those medians hard to trust), with a warning when either leg drops requests. The undici.request client is already in your matrix, so no new client was needed. Happy to adjust the PR if you'd prefer the failures reported differently. |
|
Circling back now that #1 has merged (thanks @Farenheith) — the request-failure counts it adds are exactly what make the HTTP/1.1 medians interpretable. The h1 variance in this benchmark comes from So the interesting next step is re-running the h2 comparison the same way: if the axios/undici gap there survives once refused-connection noise is isolated from genuine latency, it's a real difference worth digging into; if it moves around with the failure counts, it was measurement noise. Happy to help run that side by side. |
|
Ran the merged benchmark ( Before trusting the counter I checked it against cases with a known answer: a closed port counts every request as a failure (60/60), a healthy server counts none, a server that 500s one request in seven reports exactly 8 of 60, and the h2 legs negotiate HTTP/2 while the h1 leg negotiates HTTP/1.1. One detail worth stating plainly, because it changes how the column reads: the "median time" is the median of the per-run wall clock for the whole 300-request batch, and failed requests sit inside that wall clock. It is not a median over successful requests. HTTP/2: zero failures. Across all six executions and all five h2 clients, 0 of 360,000 requests failed. There is no refused-connection noise on the h2 side to separate out — those numbers were already measured over complete work. HTTP/1.1: undici loses 22.67%, identically, every time. 2720 is 68 x 40: exactly 68 requests refused per 300-request run, in six independent executions across two undici versions. A count that reproducible is not a burst. The cause is connection reuse, not HTTP/1.1. The h2 legs pass an explicit Raising the accept backlog changes nothing, so it is not the listen queue. axios reuses sockets and pays the cost once, on the cold batch. Giving undici a pool removes the failures entirely and serves all 1,500. The h1 leg is comparing a client that opens a socket per request against one that does not; the protocol is not what is being measured there. axios vs The same client's median moves by roughly 2x between executions on the same machine (axios 50.8 -> 95.4). That swing is larger than any gap being compared, so at 300 concurrency this harness cannot resolve a 10% difference on this hardware — the original "axios is 10.30% faster" sits inside its own noise. One correction on the stated reason for setting undici.request(url, { headers: { 'x-foo': 'bar\r\nX-Injected: 1' } })
// InvalidArgumentError: invalid x-foo header
undici.request(url, { headers: { 'bad header name': 'v' } })
// InvalidArgumentError: invalid header key
undici.request(url, { headers: ['host', 'a', 'host', 'b'] })
// InvalidArgumentError: duplicate host header
undici.request(url, { headersTimeout: 300 }) // against a server that never responds
// HeadersTimeoutError UND_ERR_HEADERS_TIMEOUT after 339msCRLF in header values, invalid header names and duplicate Two suggestions, if useful: give the h1 undici client an explicit Not verified: all of the above is one Windows machine. The 68-per-run refusal count is very likely OS specific and I would expect different absolute numbers on your Ubuntu box. The 1.00 vs 3.31 requests-per-connection asymmetry, though, is in the harness configuration rather than the OS, so it should reproduce there. The workflow has not been dispatched since April, so a CI run would now report the counts too. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Hi, I was just exploring some benchmarks to try to justify at my company whether or not to use http2 in internal microservices. We're also in a movement to remove axios usage and adopt native fetch.
Of course, http2 can only be achieved with native fetch right now by using undici agent, but soon it'll be native as well (Node 26).
All that said, I created the following benchmark:
http2-benchmark-undici.js
The results were partially unexpected for me:
I even tried to quickly write a custom Agent for Undici, but this is not so trivial, so I hung that possibility for later.
Axios setup for http2 was the default one, just setting up httpVersion, while undici agent I've set up like this:
I've tried to find a configuration where I could extract most of undici, but the most relevant one for this test was to set up connections to 1, removing pool overhead.
This is not a problem with the lib. Performance is consistently better with much less variance in response time when using http2 compared to http1, as expected, and this still justifies trying that out with my colleagues, but I wonder if there's room for improvement in HTTP/2 requests, or if this difference is justified by extra security measures undici takes
All reactions