Repository navigation
std::fs::canonicalize returns UNC paths on Windows, and a lot of software doesn't support UNC paths #42869
Description
Hi, I hope this is the right forum/format to register this problem, let me know if it's not.
Today I tried to use std::fs::canonicalize to make a path absolute so that I could execute it with std::process::Command. canonicalize returns so-called "UNC paths", which look like this: \\?\C:\foo\bar\... (sometimes the ? can be a hostname).
It turns out you can't pass a UNC path as the current directory when starting a process (i.e., Command::new(...).current_dir(unc_path)). In fact, a lot of other apps will blow up if you pass them a UNC path: for example, Microsoft's own cl.exe compiler doesn't support it: rust-lang/cc-rs#169
It feels to me that maybe returning UNC paths from canonicalize is the wrong choice, given that they don't work in so many places. It'd probably be better to return a simple "absolute path", which begins with the drive letter, instead of returning a UNC path, and instead provide a separate function specifically for generating UNC paths for people who need them.
Maybe if this is too much of an incompatible change, a new function for creating absolute paths should be added to std? I'd bet, however, that making the change to canonicalize itself would suddenly make more software suddenly start working rather than suddenly break.
Activity
retep998 commentedon Jun 23, 2017
canonicalizesimply asks the kernel to canonicalize the path, which it does and happens to return the canonical path as a root local device path. Root local device paths are capable of representing some paths which normal absolute paths are incapable of representing accurately (such as components being named ".." or "." or having "/" in them), along with the fact that they're the only way to call many system functions with paths longer thanMAX_PATH(aside from being on some recent version of Windows 10 and having a certain registry key enabled). As a result havinglibstdautomatically strip the prefix would definitely break some situations. However I'm definitely in favor of having more support inlibstdfor converting between these different kinds of paths so the user can easily turn a root local device path into an absolute path. I'd also love to have anfs::normalizewhich merely normalizes a possibly relative path into an absolute path without hitting the filesystem on Windows.oops, don't use canonicalize to get the path to webpack: rust-lang/ru…
retep998 commentedon Jun 24, 2017
In reference to your commit which referenced this PR, normalization is not the same as merely joining the path onto the current directory due to drive relative paths being relative to the current directory on the given drive. For example given a drive relative path of
C:foo, and anenv::current_dir()ofD:\bar, normalizingC:foowill have to get the current directory forC:\and could end up being normalized to something radically different such asC:\i\dont\even\foo.radix commentedon Jun 24, 2017
thanks, @retep998 :) it's just a hacked-together build tool that probably will eventually be replaced with something else, and I didn't intend to notify this ticket about my commit. but I guess it goes to show that a good way to get an absolute path in std would be really helpful.
nagisa commentedon Jun 25, 2017
Command::current_dirshould be fixed. I doubt we will change canonlicalize.Note, the i-wrong tag is only for the
Command::current_dir, not thecanonicalizebehaviour.retep998 commentedon Jun 25, 2017
Quick testing on Windows 10.0.15063 indicates that both
SetCurrentDirectoryWandCreateProcessWare okay with a current directory starting with\\?\. They are not okay with a current directory that exceedsMAX_PATHregardless of\\?\.CreateProcessWis okay with the path to the process itself starting with\\?\regardless of whether the first parameter is used.CreateProcessWis only okay with the path to the process exceedingMAX_PATHif it starts with\\?\and is specified as the first parameter which Rust does not currently use. I testedstd::process::Command::current_dirand it works as expected, accepting paths starting with\\?\but rejecting any paths exceedingMAX_PATH.kornelski commentedon Sep 5, 2017
Technically, AFAIK it is safe to strip the prefix in common simple cases (absolute path with a drive letter, no reserved names, shorter than max_path), and leave it otherwise.
So I think there's no need to compromise on correctness as far as stdlib goes. The trade-off is between failing early and exposing other software that doesn't support UNC paths vs maximizing interoperability with non-UNC software.
In an ideal world, I would prefer the "fail early" approach, so that limitations are quickly found and removed. However, Windows/DOS path handling has exceptionally long and messy history and decades of Microsoft bending over backwards to let old software not upgrade its path handling. If Microsoft can't push developers towards UNC, and fails to enforce this even in their own products, I have no hope of Rust shifting the Windows ecosystem to UNC. It will rather just frustrate Rust users and make Rust seem less reliable on Windows.
So in this case I suggest trying to maximize interoperability instead, and canonicalize to regular paths whenever possible (using UNC only for paths that can't be handled otherwise).
Also, careful stripping of the prefix done in stdlib will be much safer than other crates stripping it unconditionally (because realistically whenever someone runs into this problem, they'll just strip it unconditionally)
ofek commentedon Oct 9, 2017
@kornelski I completely agree. The current behavior is unexpected in my opinion.
danielpclark commentedon Oct 10, 2017
I hope this is helpful…
According to Microsoft:
Source: https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx
And the Ruby language uses forward slashes for File paths and that works on Windows.
kornelski commentedon Nov 22, 2017
I've looked at this problem in detail. There are a few rules which need to be checked to safely strip the UNC prefix. It can be implemented as a simple state machine.
I've implemented that using public APIs, but because
OsStris opaque it's not nearly as nice as stdlib's implementation could have been:https://lib.rs/dunce
So I'm still hoping canonicalize would do it automatically, because if it's done only for legacy-compatible paths there's no downside: all paths work for UNC-aware programs, and all paths that can work for legacy programs work too.
63 remaining items
nathaniel-daniel commentedon Jan 13, 2025
I don't think a legacy path -> UNC path conversion is possible using just the
pushfunction.This conversion requires resolving links, since if the old legacy path used links it will break if it becomes a UNC path. Those link components would be interpreted verbatim instead of resolving the link. This means a conversion like this would have to check the filesystem, a fallible operation. A
pushoperation is expected to be infallible due to its signature. I also don't think it would be advisable to have apushoperation have a chance to hit the filesystem.Maybe introducing a new
smart_pushfunction or something would work, but that's its own can of worms and doesn't resolve the backwards compatibility issue here.Also, this conversion may not even be needed in some cases. There's an option to allow legacy paths to exceed MAX_PATH, though its a bit convoluted. Cargo seems to use this option. However, like with UNC paths, these longer paths are not compatible with all APIs. I would also think that they would fail if passed to a non-long-path-aware program.
EliahKagan commentedon Jan 13, 2025
Can you give an example? If you're talking about symbolic links, those resolve fine in
\\?\paths.This works even in nontrivial cases with multiple symlinks in the same path, links to both files and directories, and where the symlinks themselves represent their targets in ways
\\?\paths themselves do not allow. As shown there, the symlinks' targets can even be relative and contain..components, even though\\?\paths must be absolute and any..components in the\\?\paths themselves would be treated literally.For symlinks, as well as any other kind of reparse point, treating it literally in the sense that is relevant to
\\?\path interpretation is completely consistent with, and does not preclude, following the link. Those two concepts are independent.But it may be that I am just not understanding what you're describing. I thought of symlinks, but you may mean something else by "links".
nathaniel-daniel commentedon Jan 13, 2025
No, I was definitely thinking about symlinks. I even remember testing this behavior before my first post here using
mklink /D link dir. But I can't replicate what I did. I think I'm simply mistaken, somehow. Thanks for catching that.Maybe
pushcould be smarter here after all. That would be nice.ofek commentedon Jan 13, 2025
The benefit of having an improved
pushimplementation is rather inconsequential as the issue is mainly about interoperability and expectations. Even something as simple as displaying a resolved path to the human, non-Windows expert user is problematic.Has there ever been a technically-breaking change that occurred in a minor Rust release because it was considered a bug fix or unintended behavior?
nathaniel-daniel commentedon Jan 13, 2025
The benefit of a smarter push would be that it would behave like normal for most paths. Only when it detects a situation where pushing a component would create a path the end user couldn't use as-is, it would convert it to UNC. So theoretically, nothing would change for cases where things aren't already broken and broken things might work in more cases.
That isn't to say it's without issues. As an example, perhaps the application is long path aware and the UNC conversion isn't necessary. Or maybe the path is being sent to a long path aware application. Or like you said, maybe its simply for display and users may be confused by the prefix (though I'd argue showing a UNC path anyways in case they decide to copy/paste it in another non-long-path-aware program and wonder why it didn't work).
Also, a smarter push would make it easier to rationalize de-UNC-ing automatically within
canonicalize.Rust has dropped breaking changes in minor releases in the past, though that's at a language-level. I don't recall any stdlib level, but I would guess that they exist. There was an issue with changing the layout of an stdlib type, but I consider the libraries to be at fault and the change was pretty well coordinated with a lot of warning. I think the idea is to minimize breaking changes and their impact.
ofek commentedon Mar 8, 2026
The behavior of this function remains a source of friction for users as you can see from all of the bug reports to large projects that continue linking back to this issue.
What is the process for proposing such a change to a standard library function? I'm unfamiliar with how the ecosystem handles these things.
madsmtm commentedon Mar 9, 2026
@ChrisDenton wrote:
So to fix this, it would procedurally be enough to file a PR with an implementation that does that, and nominate it for
t-libs-apito FCP. Normally I'd say a crater run too, but we don't have that on Windows, so we have no way to measure the impact.For a contentious topic such as this one though, I would recommend you gauge the team's opinion first by filing an ACP. In that, it'd probably be useful if you clearly compared to e.g. #59117, and maybe found examples of code in the wild that's broken by the current behaviour?
Someone probably also need to fix #135443 first (or if not, the ACP should discuss why we're not doing that).
fix: funnel all git invocations through one helper to enforce uniform…
fix: funnel all git invocations through one helper to enforce uniform…
fix(update): repair Windows MSI self-update detection
feat(core): cross-platform canonical path generator + project_fs refa…
feat(core): cross-platform canonical path generator + project_fs refa…
fix(settings): accept WSL UNC paths and fix Windows home check in con…