Repository navigation
Collecting input: validating releases and improving cache #2375
Description
I'm looking at how we can improve our end to end storage of distfiles. Part of that also touches our release process, pushing distfiles and making these public (in the eyes of a user as well as cache).
To extend my train of thought a bit; I want our end storage to be something a bit more distributed as well as "colder"; be it GCS or s3-like. This way, we can offload a lot of traffic from our server(s) as well as improve resiliency when catering to the majority of our bandwidth.
As far as I understand, we invalidate cache (read: changing user expectations) as part of release processes. One reason seems to be that some releases needs to be re-baked. What other reasons do we have? Incomplete releases perhaps?
Activity
jbergstroem commentedon Jul 3, 2020
My first idea was to have any ci output with destination "dist" landing in a staging folder. Before passing onto cold storage (read: never write twice, never delete) there would be some kind of validation. Preferably automatic (checksums, extracting, binary checks, ..) or alternatively manual since parts of our release is already intentionally manual.
Based on the input I collect this is subject to change.
rvagg commentedon Jul 3, 2020
We invalidate cache because brute-force "invalidate all" is the best tool we have and when we publish new website updates and directory indexes we need those updates to be pushed out to the edges and not stay stale. CF has more fine-grained controls now but I think it means tagging everything into groups with nginx so we can invalidate just certain things. That work hasn't been done because ... it's work, and it's not simple.
jbergstroem commentedon Jul 3, 2020
I usually read this as "I don't know what I'm changing". My dream scenario would be to understand these changes and introduce a flow where we can replace "restarting windows" with "
killall Finder".Cache tags is one solution. The landscape has changed a bit so I would say we have more options now.
Discovering all exceptions would probably help us make the best decision and Make It So.
rvagg commentedon Jul 3, 2020
Right, it's always been a crappy setup but originally we didn't have much choice and as options started appearing they were just too complicated and time consuming to implement. We have two broad cases which overlap:
The tragedy of our current situation is all of the download binaries getting invalidated. Even if we could just say "any .tar.?z, .exe, .msi, .etc should never be invalidated, you can have them for as long as you like" then it'd be a massive step forward.
jbergstroem commentedon Jul 3, 2020
This is also what I find being the biggest win (hence #2376).
I don't remember this - can you share a link?
Doable; but ultimately not scalable hand-tooling. We can set timeouts from nginx based on file-types and do exceptions for indexes. The maintainable solution here is usually having a caching server in front that more easily can express logic for these cases; be it ATS or varnish (or cloudflare or fastly). If we split out downloads, we don't have to change much of this behavior for now since cache populates quickly.
rvagg commentedon Jul 3, 2020
what I mean is that "release" type builds are special cases of all the types. nightly, test and rc just get auto-promoted and they don't interact with the rest of the website at all, they're just a new subdirectory, a change to index.json and index.tab in their respective parent directories and a refresh of that parent directory index. For release builds that are either the current "current" line or the current "lts" line, the main website page gets updated to list it, https://nodejs.org/en/download/ (and friends) gets updated to show it.
We run this in crontab every 5 minutes: https://github.com/nodejs/build/blob/master/ansible/www-standalone/resources/scripts/check-build-site.sh which checks the release build index.tab with website index.html. If the former is newer then it needs a website rebuild.
This is also very relevant, and also in need of attention if you're keen for something to chew on: #2123 (comment)
jbergstroem commentedon Jul 3, 2020
This seems highly relevant to #2359 as well.
github-actions commentedon Apr 30, 2021
This issue is stale because it has been open many days with no activity. It will be closed soon unless the stale label is removed or a comment is made.