Error when canonicalizing path on RAM drive

R:\rust-test>rustc --version --verbose
warning: could not canonicalize path: 'R:\rust-test'
warning: could not canonicalize path: 'R:\rust-test'
warning: could not canonicalize path: 'R:\'
rustc 1.24.0 (4d90ac38c 2018-02-12)
binary: rustc
commit-hash: 4d90ac38c0b61bb69470b61ea2cccea0df48d9e5
commit-date: 2018-02-12
host: x86_64-pc-windows-msvc
release: 1.24.0
LLVM version: 4.0

I created a new project using 'cargo new rust-test --bin'
Imported project to Intellij and clicked run. (without Intellij fails as well, just less text)

Console:

C:/Users/<username>/.cargo/bin/cargo.exe run --package rust-test --bin rust-test
warning: could not canonicalize path: 'R:\rust-test'
warning: could not canonicalize path: 'R:\rust-test'
warning: could not canonicalize path: 'R:\'
   Compiling rust-test v0.1.0 (file:///R:/rust-test)
error: incremental compilation: error canonicalizing path `R:\rust-test\target\debug\incremental\rust_test-1apbc64jn2fjp`: Niepoprawna funkcja. (os error 1)

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.24.0 (4d90ac38c 2018-02-12) running on x86_64-pc-windows-msvc

note: run with `RUST_BACKTRACE=1` for a backtrace

thread 'rustc' panicked at 'src\librustc\session\mod.rs:665: Trying to get session directory from IncrCompSession `NotInitialized`', src\librustc\session\mod.rs:1141:26
stack backtrace:
   0: <std::sync::condvar::WaitTimeoutResult as core::fmt::Debug>::fmt
   1: <std::time::SystemTimeError as core::fmt::Display>::fmt
   2: std::panicking::Location::column
   3: std::panicking::Location::column
   4: std::panicking::rust_panic_with_hook
   5: <alloc::vec::Vec<(alloc::string::String, u64)> as rustc::session::config::dep_tracking::DepTrackingHash>::hash
   6: rustc::ty::context::tls::span_debug
   7: rustc::session::bug_fmt
   8: rustc::session::bug_fmt
   9: rustc::session::Session::incr_comp_session_dir
  10: rustc_incremental::persist::load::load_dep_graph
  11: rustc_driver::driver::compile_input
  12: rustc_driver::run_compiler
  13: rustc_driver::profile::dump
  14: _rust_maybe_catch_panic
  15: <rustc_driver::pretty::UserIdentifiedItem as core::fmt::Debug>::fmt
  16: <std::sync::condvar::Condvar as core::default::Default>::default
  17: std::sys::windows::thread::Thread::new
  18: BaseThreadInitThunk

error: Could not compile `rust-test`.

To learn more, run the command again with --verbose.

Process finished with exit code 101

'Niepoprawna funkcja.' means 'invalid function'

R is an in-memory drive created using ImDisk program.
cargo new ... cargo run works fine on external ssd drive.
So in-memory drive is probably the cause. Reporting because console output says you will appreciate it.

I wonder whether we actually need canonicalization here. It's very likely that normalization would be enough and not cause this issue by virtue of not hitting the filesystem.

cc @rust-lang/compiler Is there any reason we need to do canonicalization, specifically the symbolic link resolution aspect? Turning relative paths into absolute paths can be done without canonicalization so if we don't actually need canonicalization...

IIRC, canonicalization was the most robust way of getting Windows to transform paths into the \\?\ form. There might have been other reasons. It has not caused a problem so far but if it does not support in-memory drives, we'll have to take another look.

I just got same error.

r:\cargo new hello-world-cargo --bin

and got this errors:

warning: could not canonicalize path: 'r:\rust_test\hello-world-cargo'
warning: could not canonicalize path: 'r:\rust_test\hello-world-cargo'
warning: could not canonicalize path: 'r:\rust_test'
warning: could not canonicalize path: 'r:\'
   Compiling hello-world-cargo v0.1.0 (R:\rust_test\hello-world-cargo)
error: incremental compilation: error canonicalizing path `r:\rust_test\hello-world-cargo\target\debug\incremental\hello_world_cargo-35pjmbew4invf`: Incorrect function. (os error 1)

thread 'rustc' panicked at 'trying to get session directory from `IncrCompSession`: NotInitialized', src\librustc_session\session.rs:736:52
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.41.0 (5e1a79984 2020-01-27) running on i686-pc-windows-msvc

note: compiler flags: -C debuginfo=2 -C incremental --crate-type bin

note: some of the compiler flags provided by cargo are hidden

error: aborting due to previous error

error: could not compile `hello-world-cargo`.

To learn more, run the command again with --verbose.

Also, R is an in-memory drive created using ImDisk program.
Copying solution to regular C:\ and build works.
Also, cargo on subst drives also works without issues.

I saw this issue linked from #79449

The issue is basically that GetFinalPathNameByHandleW fails if the drive was not mounted using the Windows Mount Manager. Some mounting tools will use it, while others will not. (Notably, Dokan has a flag that specifies if we're using the mounting manger or not.)

But you can use QueryDosDeviceW to get the NT path given a drive letter.

The procedure basically becomes to first call GetFinalPathNameByHandleW and request a DOS path. If you get a dos path, great, you're done. Otherwise, you need to request an NT path. Then you call QueryDosDeviceW on each of the 26 drive letters, and match the path prefix. If you find a match, you've canonicalized the path for a drive letter which was not mounted using the Mount Manager.

But you can use QueryDosDeviceW to get the NT path given a drive letter.

This can still fail if the drive was mounted to a directory and not a drive letter.

Absolutely true too, if the path isn't mounted as a drive letter, it won't be found using QueryDosDeviceW. But chances are good that people will probably be using mounting tools with a drive letter. If they really need to mount in a subdirectory, they can use a different mounting program that makes use of the mount manager.

Eventually you could just give up and return a path like \\?\GLOBALROOT\Device\HarddiskVolume4\Path\To\File. This kind of path will not work in the Windows Explorer shell or standard File dialogs. But it will work everywhere else. fopen will open it. Notepad will open that kind of path if you specify it from the command line. 7-Zip File Manager will even let you browse the directories and subdirectories of a path like that.

But I'm sure nobody would ever expect a "Canonicalize" function to return a path beginning with \\?\GLOBALROOT\.

Eventually you could just give up and return a path like \\?\GLOBALROOT\Device\HarddiskVolume4\Path\To\File.

Yes this has been considered before. It stalled on the "weirdness" factor though. If someone wants to push for again, I'd be minded to support them.

I would ask though, why Dokan defaults to not integrating with the mount manager? Should that not be the default? Tbh, I'm confused why that's even an option.

Using the mount manager causes Windows to immediately generate a Recycle Bin and System Volume Information folder.

I've stumbled upon this issue when trying to setup virtiofs drive.
When using winfsp this problem can be solved by using a different mount point, see here winfsp/winfsp#410

winfsp apperently also supports ram disks, so perhaps it can be useful for someone.

So here's the problem that we're trying to solve:

We have an input path.
It points to a target file on a volume that is not mounted using the mount manager.
The path works, it can be opened, and it will canonicalize as an NT path.
But it will not canonicalize as a DOS path.

For this problem:

Let's assume that our input path is already normalized, and doesn't contain any .. . or multiple backslashes.
It will canonicalize to an NT path beginning with "\Device\", followed by a device name. NT Paths that don't begin with "\Device\" should not be handled.
NT paths "\Device\Lanmanredirector" (for network drives) and "\Device\Mup" (for UNC paths) should be ignored, they should already canonicalize as DOS paths anyway.

Things about mounted volumes:

Volumes can only be mounted once
Volumes can be mounted either as a drive letter, or a directory mount point.
If you have a working drive letter which points to a device, that's the canonical mount point for the volume.
If you have a directory which is a reparse point, and the reparse point tag is "IO_REPARSE_TAG_MOUNT_POINT", that's the canonical mount point for the volume.

So again, the goal is that given a target NT path that starts with "\Device\xxxx", find the canonical mount point for that device.

For drive letter mount points:

Call QueryDosDeviceW on every drive letter
QueryDosDeviceW will return an NT path such as "\Device\HarddiskVolume4" when used with a mounted volume
On a match, the target NT Path will begin with the drive letter's NT device path. It will be a complete prefix match.
When used with a "subst" path, it will return the target directory instead (like \??\C:\file.txt), so a "subst" path can not be confused with the canonical mount point.
To speed up subsequent queries
- Store the last known NT path of all 26 drive letters.
- Check if the target NT path begins with any of our known device paths for drive letters.
- If we find a match, a drive letter could still change since the last time it was checked. So recheck that the drive letter still matches. Only one drive letter to check rather than 26.

For directory mount points:
A directory mount point could be anywhere on the filesystem. Fortunately, the input path will usually tell you how to find the mount point.

Examples

Easy: C:\Mounted Drives\mountpoint\file.txt
- We're trying to find "mountpoint".
- When we open "mountpoint" as a reparse point:
  - It must canonicalize as a DOS path
  - It has the FILE_ATTRIBUTE_REPARSE_POINT file attribute
  - It has the reparse point tag IO_REPARSE_TAG_MOUNT_POINT
- When we open "mountpoint" as a regular directory:
  - When canonicalized an an NT path, we get a path like "\Device\xxxx"
  - If your target NT path begins with that device, we've found the device, and now have the canonical DOS path of the target.
Nested: C:\Mounted Drives\mountpoint\Mounted Drives\mountpoint\file.txt
- Mount points could be nested.
Needs a reparse: C:\symlink.txt ->C:\Mounted Drives\mountpoint\file.txt
- This case requires that you read the symbolic link target and symbolic link type
  - There are both absolute symbolic links and relative symbolic links. Junctions are no different than absolute symbolic links for this purpose.
  - Reparse means reparse, once you've substituted the symbolic link for its target, start over.
Multiple reparses: C:\symlink\mountpoint\symlink\mountpoint\file.txt
Subst: Z:\Mounted Drives\mountpoint\file.txt where Z: -> C:
Weird: \\?\GLOBALROOT\Device\BootPartition\Mounted Drives\mountpoint\file.txt
- An example of a path where the left side is not canonical, but will canonicalize as a DOS path once you get far enough.
Impossible: C:\symlink.txt -> \\?\GLOBALROOT\Device\xxxx\file.txt
- This path simply doesn't have any route that reveals the mount point.

So then how to come up with an algorithm to find the mount point directory?
Do we already know it:

Cache any mount points discovered.
Check if the target NT path begins with a known mount point.
Recheck at the mount point still points to the target device

Right-to-left approach:

Needed to handle the "Subst" or "Weird" case where the left side is not canonical. Using a right-to-left approach could also avoid checking early symbolic links in the path.
It's right-to-left because we remove pieces from the right side of the path as we proceed
We stop the Right-to-left approach when we find a canonicalized path for that piece
This could just result in C:, or even nothing.
Algorithm:
Open the path as a reparse point and look at the reparse point tag.
- Attempt to canonicalize as a DOS path
  - If it succeeds, we have a canonicalized left side of the path and can stop the right-to-left search after this step.
- Regular File / Regular Directory: proceed to the parent path
- Absolute Symbolic Link: Substitute the left side of the path with the symbolic link's target, and start over
- Relative Symbolic Link: If the symbolic link refers to a parent directory, we can't do that in a right-to-left approach. We need to be at the left-to-right approach to handle those.
- Mount Point: We're looking for these. We want the one that matches the target's device path.

Left-to-right approach:

We need to start with a canonical path on the left side. If we didn't get a canonical path on the left side, just fail and don't run this.
We're adding path pieces every step.
Open the path as a reparse point and look at the reparse point tag.
- Regular File / Regular Directory: proceed to the next path
- Symbolic Link: Substitute the left side of the path with the symbolic link's target, and start over (with the right-to-left step)
- Mount Point: We're looking for these, and want the one that matches the target's device path

I think this should find a directory mount point if the path leads there.

Rollup merge of rust-lang#161951 - ChrisDenton:canonicalize-boogaloo,…

Unrolled build for #161951 in rollup 162254

Navigation Menu

Repository navigation

Error when canonicalizing path on RAM drive #48249

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Error when canonicalizing path on RAM drive #48249

Description

Activity

retep998 commented on Feb 16, 2018

retep998 commented on Feb 20, 2018

michaelwoerister commented on Feb 21, 2018

dsoronda commented on Feb 9, 2020

Dwedit commented on Aug 22, 2023

ChrisDenton commented on Aug 22, 2023

Dwedit commented on Aug 22, 2023

ChrisDenton commented on Aug 22, 2023

Dwedit commented on Aug 23, 2023

enbyted commented on May 24, 2025

Dwedit commented on May 28, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions