Repository navigation
Prepare the Worker for Production Adoption #74
Description
There are still a few loose ends we should fix before releasing our Worker fully on production, this issue tracks all pending work:
- fix: return a 400 for malformed URIs #64
- Automatically merge & deploy prs for updating redirect links #66
-
.txt.ascfiles are downloaded instead of displayed in browser #69 - We should have default strategies for caching #73
- https://nodejs.org/download/release/index.* not updated for Node 21.2.0 #72
- fix: use same directory listing format as nginx #65
- Ensure that the Promote Script is Syncing Files Correctly
- Ensure Dark Theme / Styling on Directory Listing working as expected
Staged Tests
- Wipe R2 once-again
- Resync Everything
- Remove the v21.2.0 from R2
- Run the Promote Script and ensure that we have 1:1 match from direct.nodejs.org to r2.nodejs.org
- Ensure that the new "Release" Tag on GitHub nodejs/node is triggering correctly our release
- We might be able to test all this functionality with the next v18.x LTS release (cc @targos)
Current Rollout
The items checked, mean that are now being served through Cloudflare Workers and our R2 (S3) bucket instead of our DigitalOcean Server + Cloudflare Load Balancer
-
/docs -
/api -
/download -
/metrics -
/dist
Only after all these are ack'd and confirmed we should switch traffic back to the Worker for /download and /dist
Activity
ovflowd commentedon Nov 23, 2023
So @targos afaik there's now the question why a new release did not trigger the GitHub Action, right?
And there's also the issue that when the promotion script runs... the R2 uploads start to fail due to the CPU/Memory being used a lot... Or maybe because the network traffic on that moment is huge (due to the cache being invalidate on Cloudflare?)
I feel that once we switch the traffic to R2 we wouldn't have this issue with uploading to R2 anymore, no?
ovflowd commentedon Nov 23, 2023
cc @MattIPv4 your input here would be nice. Is there a way we can make the droplet have a 2nd network interface? So that since the first one is congestioned the 2nd one can be used just for DO <> CF comms?
ovflowd commentedon Nov 23, 2023
(That would be great, because if we have a 2nd network interface with another public IP we can assign a domain there like (direct-cf.nodejs.org) and that can be also the domain that CF uses for caching things (?) idk if that would even make a difference.
MattIPv4 commentedon Nov 24, 2023
I'm not sure if it'd give you a second interface, but a Droplet can definitely have two public IPs -- the default anchor address of the Droplet itself, and then a reserved IP as well. https://docs.digitalocean.com/products/networking/reserved-ips/
MattIPv4 commentedon Nov 24, 2023
That being said, I'm not sure how having a second IP/hostname will achieve anything? As soon as that hostname becomes publicly known, folks will start abusing it just like they do the current one...
Maybe I've missed a discussion somewhere, but why isn't all HTTP traffic on the Droplet just restricted to Cloudflare only? There shouldn't be anything that needs direct access rather than being proxied through Cloudflare with some level of caching, or the ability for us to lock down access there (we could still have a hostname with little caching if needed, but have full access control in Cloudflare).
targos commentedon Nov 24, 2023
I'm not sure it's related to people accessing the server directly. When there's a new release, we just have a lot of load coming from Cloudflare servers.
One solution might be to disable concurrency in the aws S3 commands.
ovflowd commentedon Dec 3, 2023
I think we've merged the last bug-fixes on our side. Now we only need to ensure the sync script is working correctly and the links job is being triggered correctly.
@MoLow any update on that front?
flakey5 commentedon Jan 10, 2024
https://github.com/nodejs/release-cloudflare-worker/actions/runs/7475122417 - seems to be working
ovflowd commentedon Jan 10, 2024
Wdym? That action was triggered because of a commit to main on cloudflare worker 🤔
MattIPv4 commentedon Jan 10, 2024
No, that run was manually triggered by the bot, against the latest commit in main