You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Revisit aiohttp vs pycurl client bake-off on target hardware/network #62
The simpler_objects.client library is currently built on pycurl, chosen over aiohttp via the throughput bake-off in #21 (see upload-behavior-demo/bench_throughput.py). That bake-off should be revisited under conditions closer to production before the choice is treated as final.
Why revisit
The bake-off that picked pycurl (~1.45x faster upload: pycurl ~150 MiB/s vs aiohttp ~103 MiB/s) ran under conditions that do not match the deployment target:
x86 only — the target hardware is ARM. The relative cost of aiohttp's Python-level streaming vs. libcurl's C path may differ on ARM.
localhost only — the target is real hosts over 1 Gbps Ethernet, which caps throughput at ~119 MiB/s. pycurl's 150 MiB/s would be limited by the link while aiohttp's 103 MiB/s would not — narrowing the gap considerably. If ARM results or aiohttp tuning lift aiohttp past the link cap, the two would effectively tie.
aiohttp was measured with the straightforward data=open_file approach. Its upload bottleneck is reading the file in 64 KiB chunks via a thread-pool executor; a custom larger-chunk payload could close much of the gap.
What to do
Re-run bench_throughput.py on ARM target hardware.
Re-run it between two hosts over a 1 Gbps link (not localhost).
Try tuning aiohttp — a custom payload with ~1 MiB reads (must still report Content-Length; the locator requires it) — and re-measure.
Possible outcomes
If aiohttp ties or wins under realistic conditions: switch simpler_objects/client.py to aiohttp, which also restores the native asyncio support client library #21 asked for.
If the result is hardware/workload dependent: ship two client libraries (a pycurl variant and an aiohttp variant) and let callers choose.
If pycurl still clearly wins: keep the current implementation and close this issue.
Activity
Add simpler_objects.client library for upload and download