Add RHEL support to the python feature by jdputschadi · Pull Request #830 · devcontainers/features

Jeff Putsch (jdputschadi) · 2024-02-01T20:48:31Z

This PR adds RHEL support to the python feature.

CentOS 7, RHEL/Alma/Rocky Linux 8 & 9, Mariner, and Fedora should all work. Tests for all of these are included.

Note:

Currently the python feature only supports Debian/Ubuntu-based distributions with the apt based systems, after this change those systems will continue to work as will RedHat systems. There is user visible change for systems that are not explicity supported: they will get an error message stating "Linux distro XXXX not supported."

This is instead of "odd" messages about not being able to find apt-get on unsupported systems.

resolves #829

Jeff Putsch (jdputschadi) · 2024-02-01T21:26:47Z

I see the cause of the error in the "checks" and will update the submitted code to not fail.

Samruddhi Khandale (samruddhikhandale) · 2024-02-01T22:12:26Z

FYI - #577 is a known issue and tests against bookworm are currently failing. No pressure in fixing that, but it would be amazing if you could. ✨

… on Mariner systems

Jeff Putsch (jdputschadi) · 2024-02-02T05:00:54Z

Commit 4007701 should fix the previously failing checks/tests. If so, the python feature portions of #577 would also be resolved by this PR.

… assuming "python" will work

Jeff Putsch (jdputschadi) · 2024-02-02T05:30:53Z

Needed f20275f to finish addressing #577 for this the python feature.

Jeff Putsch (jdputschadi) · 2024-02-02T17:36:32Z

Darn it I see new failures -- serves me right for only testing the previous failures. I'll get the issue resolved.

managed. If so, add "--break-system-packages" to the pip install flags. This does not really breack system packages due to the setting of PYTHONUSERBASE during the install of pipx, but does get us past checks for installing python packages into the system python install.

Samruddhi Khandale (samruddhikhandale) · 2024-02-02T21:30:38Z

I don't see an option to run the tests, closing and opening this PR (attempt at fixing it)

Samruddhi Khandale (samruddhikhandale) · 2024-02-02T21:31:33Z

I don't see an option to run the tests, closing and opening this PR (attempt at fixing it)

Didn't help 😓

Not sure if ^ needs to be fixed.

Jeff Putsch (jdputschadi) · 2024-02-02T21:34:16Z

Samruddhi:

commit dad4f8d works for the base tests:

ubunto:focal
ubuntu:jammy
debian:11
debian:12
mcr.microsoft.com/devcontainers/base:debian
mcr.microsoft.com/devcontainers/base:ubunto

It should work on all the scenarios. I'm in the process of confirming that.

Jeff Putsch (jdputschadi) · 2024-02-02T21:38:18Z

Looks like I need to merge in a recent commit.

That may take a while, especially if I re-run tests on my end before pushing the result of the mrege.

Jeff Putsch (jdputschadi) · 2024-02-05T04:55:33Z

Commit 13a7f30 resolves test failures and passes all tests on my laptop. This should solve the known issues with the python feature outlined in #577.

Jeff Putsch (jdputschadi) · 2024-02-05T21:57:52Z

Samruddhi Khandale (@samruddhikhandale)

I don't understand the failure. Commit f9f1e7f contains code, from commit 4007701 , that is required for Mariner and also prevents the Mariner test case from failing due to lack of a "tk-devel" RPM package in Mariner:

        # Mariner does not have tk-devel package available, RedHat ubi8 and ubi9 do not have tk-devel
        if [ ${ID} != "mariner" ] && [ ${ID} != "rhel" ]; then
            REQUIRED_PKGS="${REQUIRED_PKGS} \
                tk-devel"
        fi

However the test log shows the mariner test failing for that reason (from the 0_test-scenarios\ \(python\).txt file in the downloaded logs):

2024-02-01T21:35:10.6118401Z 17.57 Running transaction
2024-02-01T21:35:10.6119075Z 17.59 Installing/Updating: mpfr-4.1.0-2.cm2.x86_64
2024-02-01T21:35:10.6119870Z 17.61 Installing/Updating: gawk-5.1.1-1.cm2.x86_64
2024-02-01T21:35:10.6127810Z 17.76 Running yum check-update ...
2024-02-01T21:35:10.6128493Z 17.77 Loaded plugin: tdnfrepogpgcheck
2024-02-01T21:35:10.6129084Z 17.87 Package findutils is already installed.
2024-02-01T21:35:10.6129689Z 17.87 Package gnupg2 is already installed.
2024-02-01T21:35:10.6130403Z 17.87 tk-devel package not found or not installed
2024-02-01T21:35:10.6131028Z 17.87 Error(1011) : No matching packages
2024-02-01T21:35:10.6132684Z 17.88 ERROR: Feature "Python" (Unknown) failed to install! Look at the documentation at https://github.com/devcontainers/features/tree/main/src/python for help troubleshooting this error.

The most recent test run acts as if it was run based of a commit associated with this PR that was earlier than 4007701.

I don't know how to get the tests to rerun. I suppose I could make an in-consequential change (add white-space somewhere) and push that change, but that doesn't seem right.

I'll follow any advice/direction you provide on this one.

Samruddhi Khandale (samruddhikhandale) · 2024-02-05T22:04:54Z

I don't know how to get the tests to rerun.

I triggered a rerun for failed action runs. A bit busy now, however, will take a peek at reviewing sometime later this afternoon.

Samruddhi Khandale (samruddhikhandale) · 2024-02-05T23:25:02Z

The tests failed this time due to 👇 As this test adds quite a bit of new tests, we are out of disk space. I think we'd need to configure powerful ACTIONS, i'd look into it.

2024-02-05T22:55:40.5897864Z [2024-02-05T22:55:40.589Z] #14 126.8 ERROR: Could not install packages due to an OSError: [Errno 28] No space left on device

Samruddhi Khandale (samruddhikhandale) · 2024-02-05T23:28:10Z

Jeff Putsch (@jdputschadi) Can you update https://github.com/devcontainers/features/blob/main/.github/workflows/test-pr.yaml#L7 to 👇. Let's see if it fixes it.

runs-on: devcontainer-image-builder-ubuntu

Samruddhi Khandale (samruddhikhandale)

Super cool, left some thoughts. Thank you so much!!!

Jeff Putsch (jdputschadi) · 2024-02-06T20:01:45Z

Jeff Putsch (@jdputschadi) Can you update https://github.com/devcontainers/features/blob/main/.github/workflows/test-pr.yaml#L7 to 👇. Let's see if it fixes it.
runs-on: devcontainer-image-builder-ubuntu

Samruddhi Khandale (@samruddhikhandale) : this line is in multiple places in the test-pr.yaml file. Do you want all of them changed?

Samruddhi Khandale (samruddhikhandale) · 2024-02-06T20:48:31Z

https://github.com/devcontainers/features/blob/main/.github/workflows/test-pr.yaml#L7

Good point, let's only update for https://github.com/devcontainers/features/blob/main/.github/workflows/test-pr.yaml#L70 now.

Jeff Putsch (jdputschadi) · 2024-02-06T21:24:51Z

Samruddhi Khandale (@samruddhikhandale) : Before I push a commit addressing the above notes and conversations, I have one overarching question:

Do you want the packages option from #768 included in the push (i.e. merge the change into one PR).

Samruddhi Khandale (samruddhikhandale) · 2024-02-06T21:29:18Z

Samruddhi Khandale (@samruddhikhandale) : Before I push a commit addressing the above notes and conversations, I have one overarching question:

Do you want the packages option from #768 included in the push (i.e. merge the change into one PR).

I was mostly hinting at replacing toolsToInstall with packages, but I understand that it's not possible.

So feel free to include/exclude based on your convenience and confidence. This PR is already big, we can merge that into a separate one.

Jeff Putsch (jdputschadi) · 2024-02-06T21:56:40Z

Samruddhi Khandale (@samruddhikhandale) : I think I've got the next push ready and it should address all your input. I'm running all tests locally to make sure they pass before executing the push.

The packages option will be left to the other PR.

Jeff Putsch (jdputschadi) · 2024-02-07T14:16:59Z

Commit baf1568 addresses input from Samruddhi Khandale (@samruddhikhandale) and passes all tests locally.

Samruddhi Khandale (samruddhikhandale)

We really appreciate all the efforts you take for the contribution! ✨

Left some minor comments.

Samruddhi Khandale (samruddhikhandale)

Looks wonderful, thank you so much! Appreciate it 👏

Samruddhi Khandale (samruddhikhandale) · 2024-02-09T17:59:20Z

The packages option will be left to the other PR.

Jeff Putsch (@jdputschadi) (Not to pressure you) Were you planning on submitting a PR? because I'd like to soon merge this new option due to devcontainers/images#856.

If you are not planning, then let me know. Looking forward to avoid duplicate efforts, thanks!

Jeff Putsch (jdputschadi) · 2024-02-09T18:11:49Z

The packages option will be left to the other PR.

Jeff Wilcox (@jeffwilcox) (Not to pressure you) Were you planning on submitting a PR? because I'd like to soon merge this new option due to devcontainers/images#856.

If you are not planning, then let me know. Looking forward to avoid duplicate efforts, thanks!

Possibly the wrong Jeff? The "other PR" I was referring to was #768. I was indicating that I would not include the "packages" feature that Alexander was submitting in #768 and leaving that work to him.

Samruddhi Khandale (samruddhikhandale) · 2024-02-09T18:15:53Z

Possibly the wrong Jeff?

Oops!

The "other PR" I was referring to was #768. I was indicating that I would not include the "packages" feature that Alexander was submitting in #768 and leaving that work to him.

Great, thanks for confirming!

Will merge this PR as soon as the tests are green.

Jeff Putsch added 9 commits December 18, 2023 14:32

install gpg2.22 on centos 7 when installing python

fc6cd0e

RHEL support, exisiting tests pass, new RHEL tests pass.

5e79d61

Merge branch 'main' into python-feature-rhel-support

31a5f7a

add tests, cleanup install organization

4f77b37

update testing to include RHEL tests

a1fd5b2

update testing to include RHEL tests

6fc48e3

undo addition of installing additional pip modules

11414e2

Merge branch 'main' into python-feature-rhel-support

f0c136e

Merge branch 'main' into python-feature-rhel-support

0feb316

Jeff Putsch (jdputschadi) requested a review from a team as a code owner February 1, 2024 20:48

fix errors installing os-provided Python on recent Debian systems and…

4007701

… on Mariner systems

adjust to properly use newly installed python (PYTHON_SRC) instead of…

f20275f

… assuming "python" will work

Samruddhi Khandale (samruddhikhandale) closed this Feb 2, 2024

Samruddhi Khandale (samruddhikhandale) reopened this Feb 2, 2024

Jeff Putsch and others added 3 commits February 2, 2024 13:47

merge from main

50c7d12

update check for managed python install. pass all tests.

13a7f30

Merge branch 'main' into python-feature-rhel-support

f9f1e7f

Samruddhi Khandale (samruddhikhandale) requested changes Feb 5, 2024

View reviewed changes

add "packages" option from PR devcontainers#768

dffe200

Jeff Putsch added 2 commits February 6, 2024 14:02

remove "packages" option

1473900

Address PR feedback, passes all tests locally.

baf1568

Samruddhi Khandale (samruddhikhandale) requested changes Feb 7, 2024

View reviewed changes

Comment thread .github/workflows/test-pr.yaml

Comment thread src/python/install.sh Outdated

Comment thread src/python/install.sh Outdated

Comment thread src/python/install.sh Outdated

Comment thread test/python/scenarios.json Outdated

Comment thread src/python/install.sh Outdated

Jeff Putsch and others added 2 commits February 7, 2024 20:46

fix install error on centos

6c1335a

Merge branch 'main' into python-feature-rhel-support

12d2e62

Samruddhi Khandale (samruddhikhandale) approved these changes Feb 9, 2024

View reviewed changes

Samruddhi Khandale (samruddhikhandale) linked an issue Feb 9, 2024 that may be closed by this pull request

[debian bookworm] Python Feature fails to install with 'error: externally-managed-environment' #577

Closed

Samruddhi Khandale (samruddhikhandale) merged commit 30e03b6 into devcontainers:main Feb 9, 2024

Jeff Putsch (jdputschadi) deleted the python-feature-rhel-support branch February 10, 2024 23:31

Uh oh!

Conversation

Jeff Putsch (jdputschadi) commented Feb 1, 2024

Uh oh!

Jeff Putsch (jdputschadi) commented Feb 1, 2024

Uh oh!

Samruddhi Khandale (samruddhikhandale) commented Feb 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Jeff Putsch (jdputschadi) commented Feb 2, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Jeff Putsch (jdputschadi) commented Feb 2, 2024

Uh oh!

Jeff Putsch (jdputschadi) commented Feb 2, 2024

Uh oh!

Samruddhi Khandale (samruddhikhandale) commented Feb 2, 2024

Uh oh!

Samruddhi Khandale (samruddhikhandale) commented Feb 2, 2024

Uh oh!

Jeff Putsch (jdputschadi) commented Feb 2, 2024

Uh oh!

Jeff Putsch (jdputschadi) commented Feb 2, 2024

Uh oh!

Jeff Putsch (jdputschadi) commented Feb 5, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Jeff Putsch (jdputschadi) commented Feb 5, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Samruddhi Khandale (samruddhikhandale) commented Feb 5, 2024

Uh oh!

Samruddhi Khandale (samruddhikhandale) commented Feb 5, 2024

Uh oh!

Samruddhi Khandale (samruddhikhandale) commented Feb 5, 2024

Uh oh!

Samruddhi Khandale (samruddhikhandale) left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Jeff Putsch (jdputschadi) commented Feb 6, 2024

Uh oh!

Samruddhi Khandale (samruddhikhandale) commented Feb 6, 2024

Uh oh!

Jeff Putsch (jdputschadi) commented Feb 6, 2024

Uh oh!

Samruddhi Khandale (samruddhikhandale) commented Feb 6, 2024

Uh oh!

Jeff Putsch (jdputschadi) commented Feb 6, 2024

Uh oh!

Jeff Putsch (jdputschadi) commented Feb 7, 2024

Uh oh!

Samruddhi Khandale (samruddhikhandale) left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Samruddhi Khandale (samruddhikhandale) left a comment

Choose a reason for hiding this comment

Uh oh!

Samruddhi Khandale (samruddhikhandale) commented Feb 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Jeff Putsch (jdputschadi) commented Feb 9, 2024

Uh oh!

Samruddhi Khandale (samruddhikhandale) commented Feb 9, 2024

Uh oh!

Reviewers

Samruddhi Khandale (samruddhikhandale) commented Feb 1, 2024 •

edited

Loading

Jeff Putsch (jdputschadi) commented Feb 2, 2024 •

edited

Loading

Jeff Putsch (jdputschadi) commented Feb 5, 2024 •

edited

Loading

Jeff Putsch (jdputschadi) commented Feb 5, 2024 •

edited

Loading

Samruddhi Khandale (samruddhikhandale) commented Feb 9, 2024 •

edited

Loading