You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We keep a record for each download instead of summarizing. On the live instance this counts feature is especially slow to persist and initialize -- we should improve it to reduce memory usage and time spent in restarts.
It has come up multiple times now, including in the discussions surrounding haskellfoundation/tech-proposals#67, that the download count feature incurs a lot of performance overhead, much more than the utility it provides.
I am interested in helping with this. Is there a fundamental reason it is difficult to optimize the storage of download counts, vs. simply removing the feature altogether?
Perhaps with some work we can optimize this feature, but my view is that it is not useful at all. What does it measure?
For example, base has ~70000 downloads total and ~200 downloads in the past month. Which is a low number, but it's actually very surprising that it is non-zero: obviously no human at all ever downloads base.tar.gz. This is all bot activity.
As another data point, aeson has ~600000 downloads total and ~250 downloads in the past month. Which is more than base, but still implausible. Every CI build on GitHub of any package depending on aeson (which is like pretty much every package) would have to download aeson.tar.gz. OK, there is some caching on GitHub Actions side, but still I alone probably trigger more than 250 downloads a month. The numbers are so low because this load is split across CDNs, Hackage mirrors and Stackage to an extent that Hackage number is hardly representative of anything.
I'd say that the number of reverse dependencies is much better predictor of package's quality than downloads count.
When I look at download counts, I don't necessarily care about the correctness, but relative scales. For example, the packages related to beam correctly show that practically no one uses beam-mysql, especially compared to beam-postgres.
Removing the download count is easy, but it's kind of final. Optimizing the feature may not be that much more work, and I'm willing to do it!
Regarding correctness of the downloads in the face of a CDN: other websites do it (e.g. https://crates.io/). I think there's a way to do this relatively simply:
I support the redirect scheme. I proposed it before but we didn't proceed out of an overabundance of caution.
As to the clunkiness of the feature, I was especially worried about it when we had hackage problems due to fd exhaustion, since download counts as implemented now open a ton of fds. However, I think we pinpointed those problems as due to other sorts of lag and slowness and not the download recalculations. Its still a weird clunky feature implementation, but some experiments at NYHac revealed it may not actually be a core performance/stability block so much as just our overall use of acid state. (cf #1463)
Activity
LaurentRDC commentedon Aug 12, 2026
It has come up multiple times now, including in the discussions surrounding haskellfoundation/tech-proposals#67, that the download count feature incurs a lot of performance overhead, much more than the utility it provides.
I am interested in helping with this. Is there a fundamental reason it is difficult to optimize the storage of download counts, vs. simply removing the feature altogether?
Related issue: #1276
Edit: After some investigation, it looks like there's two ways this can be optimized:
LaurentRDC commentedon Aug 12, 2026
cc @Bodigrim who has thought about this before as well
Bodigrim commentedon Aug 12, 2026
Perhaps with some work we can optimize this feature, but my view is that it is not useful at all. What does it measure?
For example,
basehas ~70000 downloads total and ~200 downloads in the past month. Which is a low number, but it's actually very surprising that it is non-zero: obviously no human at all ever downloadsbase.tar.gz. This is all bot activity.As another data point,
aesonhas ~600000 downloads total and ~250 downloads in the past month. Which is more thanbase, but still implausible. Every CI build on GitHub of any package depending onaeson(which is like pretty much every package) would have to downloadaeson.tar.gz. OK, there is some caching on GitHub Actions side, but still I alone probably trigger more than 250 downloads a month. The numbers are so low because this load is split across CDNs, Hackage mirrors and Stackage to an extent that Hackage number is hardly representative of anything.I'd say that the number of reverse dependencies is much better predictor of package's quality than downloads count.
LaurentRDC commentedon Aug 13, 2026
When I look at download counts, I don't necessarily care about the correctness, but relative scales. For example, the packages related to beam correctly show that practically no one uses
beam-mysql, especially compared tobeam-postgres.Removing the download count is easy, but it's kind of final. Optimizing the feature may not be that much more work, and I'm willing to do it!
Regarding correctness of the downloads in the face of a CDN: other websites do it (e.g. https://crates.io/). I think there's a way to do this relatively simply:
Cache-Control: no-storeheader for the download endpoint;This results in every download from Hackage to count in the download statistics, while the payloads themselves remain cached.
For this to work, we need the feature to be optimized. So I suggest two pieces of work:
gbaz commentedon Aug 20, 2026
I support the redirect scheme. I proposed it before but we didn't proceed out of an overabundance of caution.
As to the clunkiness of the feature, I was especially worried about it when we had hackage problems due to fd exhaustion, since download counts as implemented now open a ton of fds. However, I think we pinpointed those problems as due to other sorts of lag and slowness and not the download recalculations. Its still a weird clunky feature implementation, but some experiments at NYHac revealed it may not actually be a core performance/stability block so much as just our overall use of acid state. (cf #1463)
Batch download counts
Batch download counts
Batch download counts
Batch download counts
Batch download counts (haskell#1518)