Skip to content

Skipped raised by a conftest of a command-line path crashes pytest instead of skipping #15141

Closed as duplicate

Description

@fangchenli
  • a detailed description of the bug or problem you are having
  • output of pip list from the virtual environment you are using
  • pytest and operating system versions
  • minimal example if possible

Investigated and drafted with help from Claude Code

Problem

When a path or node id is given on the command line, pytest imports the conftests along that path while parsing the configuration ("initial conftests"), before a session exists. If one of those conftests skips at module level (pytest.importorskip(...) or pytest.skip(..., allow_module_level=True)), the Skipped exception escapes as a raw traceback and pytest exits 1.

The same conftest, imported during collection in a run without path arguments, skips its directory cleanly. So whether a directory is skipped or the whole run crashes depends only on how pytest was invoked.

Reproduction

tests/opt/conftest.py:
    import pytest
    mod = pytest.importorskip("no_such_module_xyz")

tests/opt/test_a.py:
    def test_a():
        pass

tests/test_b.py:
    def test_b():
        pass
invocation (-p no:cacheprovider) result
pytest 1 passed, 1 skipped, exit 0
pytest tests 1 passed, 1 skipped, exit 0
pytest tests/opt traceback ending Skipped: could not import 'no_such_module_xyz', exit 1
pytest tests/opt/test_a.py::test_a same traceback, exit 1

Tail of the traceback:

  File ".../tests/opt/conftest.py", line 2, in <module>
    mod = pytest.importorskip("no_such_module_xyz")
  File ".../_pytest/outcomes.py", line 271, in importorskip
    raise skipped
Skipped: could not import 'no_such_module_xyz': No module named 'no_such_module_xyz'

pytest tests only works because, for each argument, the initial conftests are those of the argument's directory and of its test* subdirectories, and opt does not match test*. Rename opt to test_opt and pytest tests crashes too.

Real-world trigger: pandas/tests/io/pytables/conftest.py does tables = pytest.importorskip("tables"), so pytest pandas/tests/io/pytables/test_x.py::test_y crashes in an environment without PyTables while pytest pandas skips the directory.

Cause

  • PytestPluginManager._set_initial_conftests calls _loadconftestmodules for each command-line anchor, and _importconftest wraps import errors with except Exception as e: raise ConftestImportFailure(...), which _main turns into a readable "ImportError while loading conftest" message and exit 4.
  • OutcomeException (and so Skipped) derives from BaseException, so it bypasses except Exception and nothing catches it.
  • During collection, pytest_make_collect_report in _pytest/runner.py treats Skipped from a collector as a skip outcome, which is why the no-argument run works.

Expected

An argument under a skipping conftest should behave like the full run: the directory is reported as skipped, and nothing crashes. For the reproduction, pytest tests/opt and pytest tests/opt/test_a.py::test_a should both report 1 skipped.

I have a patch and would open a PR if the direction is acceptable. It has three small parts:

  1. In _set_initial_conftests, catch Skipped around the per-anchor _loadconftestmodules call and leave the directory unloaded. Collection then imports the conftest again inside its collector, where Skipped is already handled, so the outcome matches the run without arguments. --help and pytest.exit() in a conftest are unchanged.
  2. In Session.collect, when a collector on the way to an argument is skipped, count the argument as accounted for. Today a node id into a module that skips at module level (pytest test_mod.py::test_a) reports the skip but also fails with ERROR: found no collectors for ..., exit 4; with this it reports 1 skipped, like pytest test_mod.py does. Collectors that fail to collect keep the usage error, as the existing acceptance tests require.
  3. In _import_module_using_spec (--import-mode=importlib), remove the module from sys.modules when its execution raises, as importlib itself does. Without this, the second import silently returns the half-executed conftest and the tests that should be skipped run instead.

Related: #4479 (closed in 2018, "importorskip can't be supported soundly inside of conftest") and #12371 (open, module-level skip in conftest skips the whole package; suggestion to deprecate import-time skips). This report is narrower: whatever the intended semantics, a bare traceback and exit 1 is not a deliberate outcome, and the run without arguments already accepts the skip. If the preferred resolution is instead to reject Skipped from a conftest with a clear usage error, that would also fix the crash; pytest.fail() in a conftest (Failed, also a BaseException) escapes the same way and could be treated alike.

Versions and environment

  • pytest 9.1.1 (also reproduced on main at fd17fdf, 2026-10-05)
  • Python 3.14.5, macOS 15 (Darwin 25.6.0)
  • pip list of the minimal environment used for the reproduction:
Package    Version
---------- -------
iniconfig  2.3.0
packaging  26.3
pluggy     1.6.0
pygments   2.21.0
pytest     9.1.1

Activity

  1. RonnyPfannschmidt commented on Oct 6, 2026

    @RonnyPfannschmidt
    Member

    🤖 Written by Claude Opus 5.5 via Claude Code for the pytest maintainers; I prompted it, it did the work, I read it.

    Thanks for the thorough analysis. Closing in favour of #15142, which takes your direction for importorskip: it is supported in conftests and skips the directory consistently, including for initial conftests (your parts 1 and 2), and the --import-mode=importlib sys.modules cleanup (part 3) is in scope there too. Other outcome exceptions at conftest import (skip(), fail(), xfail()) become a usage error. A PR against #15142 is welcome.


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions