citgm-smoker-nobuild seems broken

https://ci.nodejs.org/job/citgm-smoker-nobuild/nodes=rhel8-x64/1548/console fails with:

$ curl -O 'https://nodejs.org/download/release/<!DOCTYPE html><html><head><title>Index of downloadrelease<title><meta name='\''viewport'\'' content='\''width=device-width, initial-scale=1.0'\'' ><meta charset='\''utf-8'\'' ><style type='\''textcss'\''>td { padding-right: 16px; text-align: right; font-family: monospace }td:nth-of-type(1) { text-align: left; overflow-wrap: anywhere }td:nth-of-type(3) { white-space: nowrap } th { text-align: left; } @media(prefers-color-scheme: dark) { body { color: white; background-color:#1c1b22; } a { color: #3391ff; } a:visited { color: #C63B65; } }<style><head><body><h1>Index of downloadrelease<h1><table>

I am guessing it's related to changes to what https://nodejs.org/download/release/ returns - I notice the webpage is now styled.

FWIW the job is running:

cd $WORKSPACE && rm -rf *
if [ -n "$DOWNLOAD_LOCATION" ]; then
	DOWNLOAD_DIR="https://nodejs.org/download/$DOWNLOAD_LOCATION/"
else
	DOWNLOAD_DIR="https://nodejs.org/download/release/"
fi
LINK=`curl $DOWNLOAD_DIR | grep $NODE_VERSION | sort -t. -k 1,1n -k 2,2n -k 3,3n | tail -1 | cut -d\" -f 2 | tr -d /` 
case $nodes in
  *-ppcle|*-ppc64le) OS=linux; ARCH=ppc64le; EXT=tar.gz;;
  ubuntu*|debian*|fedora*|centos*|rhel*-x64) OS=linux; ARCH=x64; EXT=tar.gz;;
  osx*) OS=darwin; ARCH=x64; EXT=tar.gz;;
  *-s390x) OS=linux; ARCH=s390x; EXT=tar.gz;;
  aix*) OS=aix; ARCH=ppc64; EXT=tar.gz;;
  win*) OS=win; ARCH=x64; EXT=zip;;
esac
curl -O "$DOWNLOAD_DIR$LINK/node-$LINK-$OS-$ARCH.$EXT"
case $nodes in
  win*) unzip node-$LINK-$OS-$ARCH.$EXT ;;
  *) gzip -cd node-$LINK-$OS-$ARCH.$EXT | tar xf - ;;
esac
mv node-$LINK-$OS-$ARCH node

@ovflowd

/cc @ovflowd @flakey5

Quick stupid question, but are really making a GET request with a huge html snippet in the URL?

Or what exactly is the issue here?

Quick stupid question, but are really making a GET request with a huge html snippet in the URL?

+1 to that and also is this relying on parsing whatever html is returned?

(Just trying to understand whay the script is trying to do and what is failing here)

Quick stupid question, but are really making a GET request with a huge html snippet in the URL?

+1 to that and also is this relying on parsing whatever html is returned?

The html looks like the index of that path, maybe something is wrong and accidentally it is trying to send the html, I have no idea. If someone can explain what this script is doing and what it is supposed to do

The script does this:

curl https://nodejs.org/download/release/ | grep v20 | sort -t. -k 1,1n -k 2,2n -k 3,3n | tail -1 | cut -d\" -f 2 | tr -d /

You can compare with direct:

curl https://direct.nodejs.org/download/release/ | grep v20 | sort -t. -k 1,1n -k 2,2n -k 3,3n | tail -1 | cut -d\" -f 2 | tr -d /

@flakey5

The script does this:


curl https://nodejs.org/download/release/ | grep v20 | sort -t. -k 1,1n -k 2,2n -k 3,3n | tail -1 | cut -d\" -f 2 | tr -d /

The script might need to be adjusted to accommodate the new html structure of our directory listing pages.

Heh, @flakey5 I did say somewhere we were relying on the format of the directory listing k

I suspect the existing script is relying on line breaks in the html listings, but the new page has everything on a single line.

FWIW https://ci.nodejs.org/job/node-test-node-addon-api-new/ does something similar and also failing.

It would be unfair (and a lot of time lost in total) to expect everyone who was parsing nginx output to change their scripts if there's a way to fix that in the new infra.

I'm confused -- I thought https://nodejs.org/download/release/ was still supposed to be served from the DO/Joyent servers and not R2 so shouldn't have changed?

It would be unfair (and a lot of time lost in total) to expect everyone who was parsing nginx output to change their scripts if there's a way to fix that in the new infra.

Oh yeah, definitely agree with you here.

I'm confused -- I thought https://nodejs.org/download/release/ was still supposed to be served from the DO/Joyent servers and not R2 so shouldn't have changed?

Only /dist is still served from DO/Joyent. Other routes are served from R2 since Nov 8.

@targos

I suspect the existing script is relying on line breaks in the html listings, but the new page has everything on a single line.

+1 I think this is what it's relying on. Formatted the html response and got it to return this

When using the test command @targos pointed out it returns just v20.9.0

@targos

I suspect the existing script is relying on line breaks in the html listings, but the new page has everything on a single line.

+1 I think this is what it's relying on. Formatted the html response and got it to return this

When using the test command @targos pointed out it returns just v20.9.0

Do we have any updates here? Or issue to keep track of this work on the worker side of things?

Working on a fix on the worker side of things here nodejs/release-cloudflare-worker#65

@flakey5

@flakey5 good to hear you are working on it. Let me know when the fix is in place so that I can confirm it fixes the Node-api tests which were also broken (https://ci.nodejs.org/view/x%20-%20Abi%20stable%20module%20API/job/node-test-node-addon-api-new/).

The jobs should be fixed now. We moved the worker off of /download until nodejs/release-cloudflare-worker#74 is resolved due to multiple issues that we've been seeing

Well, can we manually test that they will not break once we move them back? We just merged the new directory listing can we check that?

Once nodejs/release-cloudflare-worker#76 lands this issue should be fixed on the worker's end

Ran the command against the worker again and it works

This specific issue is fixed. If something needs to be changed long-term, please open an issue and suggest what / how

Navigation Menu

Repository navigation

citgm-smoker-nobuild seems broken #1028

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

citgm-smoker-nobuild seems broken #1028

Description

Activity

richardlau commented on Nov 13, 2023

targos commented on Nov 13, 2023

ovflowd commented on Nov 13, 2023

ovflowd commented on Nov 13, 2023

flakey5 commented on Nov 13, 2023

ovflowd commented on Nov 13, 2023

ovflowd commented on Nov 13, 2023

targos commented on Nov 13, 2023

targos commented on Nov 13, 2023

ovflowd commented on Nov 13, 2023

richardlau commented on Nov 13, 2023

targos commented on Nov 13, 2023

richardlau commented on Nov 13, 2023

ovflowd commented on Nov 13, 2023

targos commented on Nov 13, 2023

flakey5 commented on Nov 13, 2023

ovflowd commented on Nov 13, 2023

flakey5 commented on Nov 13, 2023

mhdawson commented on Nov 15, 2023

flakey5 commented on Nov 15, 2023

ovflowd commented on Nov 15, 2023

flakey5 commented on Nov 15, 2023

flakey5 commented on Nov 23, 2023

targos commented on Dec 23, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions