Repository navigation
Discuss state of r2 migration on Build WG meeting #3508
Description
Opening this issue just to track this in the meeting.
we are pretty much done with getting https://r2.nodejs.org/dist/ working.
we want to make sure we did not miss anything and discuss the next steps - mainly when we should pull the plug on the migration
the Cloudflare team asked for an estimated date so they can prepare (they are depending on https://nodejs.org/dist for installing node on their infra :), they want to avoid circular reference issue )
Activity
richardlau commentedon Oct 3, 2023
FYI unless #3503 changes the meeting times, the next scheduled Build WG meeting is in the inconvenient-for-everybody slot on Wednesday 18th October. The next meeting after that, in the earlier slot, is on Tuesday 7th November.
I guess one question is whether we want to migrate before or after Node.js 21 -- usually a semver major release results in a spike in traffic.
ovflowd commentedon Oct 3, 2023
Oh yeah waiting till November is definitely a no go IMO. It would make sense to discuss about R2 way before that.
mhdawson commentedon Oct 6, 2023
I just got an error when trying to hit the link mentioned
mhdawson commentedon Oct 6, 2023
Does it make any sense to switch over a portion of our downloads first to validate on a smaller subset versus everything. Is that possible/worth any additional work.
mhdawson commentedon Oct 6, 2023
When I hit the page again, it now seems to work instead of the error
ovflowd commentedon Oct 6, 2023
@flakey5 ☝️
flakey5 commentedon Oct 6, 2023
Looking into it now to see what happened
As in, for example, requests going to
/dist/...hit the worker and all requests going to/download/...hit the origin? In that case it's definitely possible and would just be up to the routing config. For deeper paths like/dist/latesthitting the worker while/dist/vX.X.Xhits the origin it should be possible, but I'm not 100% sure on that.flakey5 commentedon Oct 7, 2023
Believe I know what the error is, should be fixed in nodejs/release-cloudflare-worker#44
mhdawson commentedon Oct 10, 2023
Something like that. Roll out in smaller steps than a big bang. Do you have a suggestion for a first good step?
richardlau commentedon Oct 10, 2023
I'm running into errors trying to promote Node.js 18.18.1 😞:
The errors are very slow -- presumably the uploads are being attempted and erroring one by one.
This is just running
tools/release.shas per https://github.com/nodejs/node/blob/main/doc/contributing/releases.md#15-promote-and-sign-the-release-builds.flakey5 commentedon Oct 10, 2023
First step could be
/metrics, that way if anything does go wrong with how we route to the worker we can fix it without disrupting too much@MoLow
richardlau commentedon Oct 10, 2023
FYI I've had to comment out the calls to
upload_to_cloudflare.shfrom the promotion scripts so thattools/release.shcan actually sign the releases (the errors from the upload were causing the release script to bail out).ovflowd commentedon Oct 10, 2023
Why would there be SSL validation failures? @flakey5 do you want to share this with the Cloudflare team? Afaik we have one Cloudflare Engineer and our Account Manager on the #nodejs-build channel
jasnell commentedon Oct 10, 2023
Check the certs provided by the server. As a temporary workaround also try setting allow unauthorized to skip the cert validation. It's possibly with either a mismatch in the cname expected by the cert or a CA issue.
40 remaining items
build: use rclone instead of aws CLI
build: use rclone instead of aws CLI
build: use rclone instead of aws CLI
build: use rclone instead of aws CLI
build: use rclone instead of aws CLI
targos commentedon Jan 22, 2025
nodejs/release-cloudflare-worker#223 may be the last thing to finish before we can close this issue.
flakey5 commentedon Jan 28, 2025
nodejs/release-cloudflare-worker#223 has been merged and is working except for a small bug that'll be fixed in nodejs/release-cloudflare-worker#274.
After that, we should be able to move the
/distand/docspaths over to the worker!build: use rclone instead of aws CLI