Skip to content

isolated_filesystem() is not thread safe #3501

Description

@kdeldycke

This is a follow-up on #2899, to cover for @Rowlando13's concerns that isolated_filesystem() is not thread safe: #2899 (comment)

Activity

  1. added this to the 8.5.0 milestone on May 22, 2026
  2. davidism commented on May 23, 2026

    @davidism
    Member

    Reminder that we should probably deprecate isolated_filesystem, there's a discussion about that somewhere around here.

  3. kdeldycke commented on May 23, 2026

    @kdeldycke
    CollaboratorAuthor

    Reminder that we should probably deprecate isolated_filesystem, there's a discussion about that somewhere around here.

    Found it at: #3123 (comment)

  4. kdeldycke commented on May 23, 2026

    @kdeldycke
    CollaboratorAuthor

    BTW it looks like the original proposal by @bphunter1972 in #3123 of a runner.set_filesystem context manager seems to address the thread-safety as each runner will produce its artefacts and assets in its own directory on CLI invokation.

  5. deleted a comment from Aniketiitk21 on May 24, 2026
  6. davidism commented on May 29, 2026

    @davidism
    Member

    To clarify: tox parallel is for running multiple interpreter environments in parallel. It is not the same as running the tests in one interpreter in parallel. I've never had an expectation that tests should be thread safe. If you were to use pytest-xdist it would use processes to run subsets of tests serially.

    For free threading, I'd expect test runners (unittest, pytest, etc) to provide a way to manage threads and locks for unsafe tests. That's getting way outside the scope of Click and this function.

    I can't tell what the comment from @Rowlando13 was saying, given the above. There's also no more information in this issue. We also all agree this function should be deprecated. Therefore I think this issue should be closed.

  7. Rowlando13 commented on May 29, 2026

    @Rowlando13
    Member

    My comment was specifically that I thought that isolated_filesystem creates a folder and then makes expectations about what is in that folder, so if there are multiple isolated_filesystem calls in a test suite and the tests are running in parallel there is a race condition between those tests.

  8. Rowlando13 commented on May 29, 2026

    @Rowlando13
    Member

    I agree with deprecating isolated_filesystem. Better to show a pytest example.

  9. Rowlando13 commented on May 29, 2026

    @Rowlando13
    Member

    I closed the related issue. I see why this should be separate and stay open if only to clarify a bit. Even with separate interpreters, I don't see how tox -p is safe(since each suite at various times make random folders, does stuff, and looks in them) unless tox runs the interpreters from different base folders.

  10. davidism commented on May 29, 2026

    @davidism
    Member

    isolated_filesystem calls tempfile.mkdtemp, so every invocation is a separate folder. It does call os.chdir, which isn't thread safe, but serializing that should be left to the test framework, not us. tox p does create a separate base dir for every environment.

  11. Rowlando13 commented on May 29, 2026

    @Rowlando13
    Member

    Thanks for explaining. Closing issue.

  12. modified the milestones: 8.5.0, 9.0.0 on Jul 8, 2026
  13. modified the milestones: 9.0.0, 8.5.0 on Jul 8, 2026
  14. kdeldycke commented on Jul 8, 2026

    @kdeldycke
    CollaboratorAuthor

    Given #3676 , I guess the milestone change to 9.0.0 is just the blast radius of the recent main <-> stable dance. So I retarget this to 8.5.0 unless an explicit decision is made.

  15. kdeldycke commented on Jul 17, 2026

    @kdeldycke
    CollaboratorAuthor

    I agree with deprecating isolated_filesystem. Better to show a pytest example.

    I created a dedicated issue at #3700

  16. locked as resolved and limited conversation to collaborators on Aug 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    test runnerCliRunner and Click's test utilities feature

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions