Collecting many file arguments in the same directory re-collects the parent directory once per argument (O(args × siblings); Windows adds a samefile per sibling) #15221

What's the problem

When many test files from the same directory are passed explicitly on the command line (e.g. a tool that selects "related" tests and runs pytest a/test_1.py a/test_2.py … a/test_63.py), collection time grows with (number of file arguments) × (number of entries in their parent directory), even though each file is only collected once.

In Session.collect (pytest 9.1.1), each file argument walks its path from the root. For the last level, handle_dupes is False when the remaining part is a single file path ("files given directly multiple times on the command line should not be deduplicated"), so _collect_one_node skips the collection cache for that call. The parent Dir is therefore collected again for every file argument: a node is built for every sibling (hook calls such as pytest_ignore_collect / pytest_collect_file for each entry), and then all but one are pruned.

On Windows the pruning step makes it worse: for every sibling whose path does not match, samefile_nofollow is called (the #11895 short-path fallback), i.e. two lstat calls per non-matching sibling per argument.

Measurements (one real-world repo)

Windows 11, Python 3.11.15, pytest 9.1.1, pytest-xdist. Warm tree, single process, --collect-only, 122 file arguments of which 63 are in one directory with 141 entries:

Without cProfile, 315 file arguments: 55.7 s → 20.7 s. The collected items (IDs and order) are identical before and after (checked on 9,608 and 38,514 items). Passing a directory instead of files does not trigger this (Session.collect ≈ 0.06 s).

Minimal reproduction (sketch)

mkdir -p repro/tests && cd repro
for i in $(seq 1 150); do printf 'def test_x():\n    pass\n' > tests/test_$i.py; done
time pytest --collect-only -q tests                       # fast
time pytest --collect-only -q $(ls tests/test_*.py | head -100)   # grows with 100 × 150

Measured with exactly this repro on Windows 11 / Python 3.11.15 / pytest 9.1.1 (-p no:cacheprovider -p no:xdist, 3 rounds, on a busy machine): directory argument 0.97–1.25 s, 100 file arguments 6.53–7.52 s for 100 trivial tests. (The gap is larger on Windows because of the per-sibling samefile fallback.)

Workaround we use

A pytest_ignore_collect hook in conftest.py that, when every argument is an existing file path (no --pyargs, no plain directory arguments), ignores entries that are neither an argument nor an ancestor of one. Since pytest discards those nodes anyway, the collected items do not change. It cannot reach the rootdir level (no conftest there) or the argument-vs-argument comparisons within the same directory, which is why we think the fix belongs upstream.

Possible fixes (suggestions)

1. Reuse the collected parent Dir result across file arguments (cache the parent level even when the last part is a file), while still not deduplicating the file itself, so the "same file given twice runs twice" behavior is kept.
2. In the pruning loop, compare by name (or os.path.normcase) first and only fall back to samefile when the names could plausibly refer to the same file (e.g. when an argument looks like a Windows short path).

Related: #11895 (the Windows short-path fallback), #13420 (a different collection slowdown with many paths, fixed by caching _check_initialpaths_for_relpath).