Issue with rust-toolchain file

[BlinkyStitt]

My project has a rust-toolchain file in its root for a nightly toolchain, but ./setup.sh simply calls "cargo" so it built with a different version than my project. This lead to the following error when running analyze.sh.

$ ~/src/siderophile/analyze.sh myapp
trawling source code of dependencies for unsafety
   Compiling libc v0.2.60

...


error[E0514]: found crate `cached` compiled by an incompatible version of rustc myapp                                   
  --> src/lib.rs:15:1
   |
15 | extern crate cached;
   | ^^^^^^^^^^^^^^^^^^^^
   |
   = help: please recompile that crate using this compiler (rustc 1.37.0-nightly (8aa42ed7c 2019-06-24))
   = note: the following crate versions were found:
           crate `cached` compiled by rustc 1.36.0 (a53f9df32 2019-07-03): ~/src/myapp/target/debug/deps/libcached-b5e2b19b56d13d52.rmeta

error: aborting due to previous error

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Cargo("Could not compile `myapp`.")', src/libcore/result.rs:1051:5
stack backtrace:
   0: std::panicking::default_hook::{{closure}}
   1: std::panicking::default_hook
   2: std::panicking::rust_panic_with_hook
   3: std::panicking::continue_panic_fmt
   4: rust_begin_unwind
   5: core::panicking::panic_fmt
   6: core::result::unwrap_failed
   7: siderophile::main
   8: std::rt::lang_start::{{closure}}
   9: std::panicking::try::do_call
  10: __rust_maybe_catch_panic
  11: std::rt::lang_start_internal
  12: main

To fix this problem, I had to build and run siderophile with the correct toolchain selected.

cd ~/src/siderophile
RUSTUP_TOOLCHAIN=$(cat ~/src/myapp/rust-toolchain) cargo build --release

$ cd ~/src/myapp
$ RUSTUP_TOOLCHAIN=$(cat rust-toolchain) ~/src/siderophile/analyze.sh myapp

trawling source code of dependencies for unsafety
   Compiling libc v0.2.60
...
generating LLVM bitcode for the callgraph
...

A note about this should probably be added to the readme.

[rozbb]

Noted. I actually ran into this issue myself on a project. I will update the README.

Do you think siderophile should respect rust-toolchain? If so, do you have any suggestions on how to make that work, since the override works on the rustup level?

[disconnect3d]

Just to give some more context. The rust-toolchain overrides the toolchain version that will be used for compiling; this is documented here: https://doc.rust-lang.org/stable/edition-guide/rust-2018/rustup-for-managing-rust-versions.html#managing-versions.

However, when you create a crate A which then uses an external crate B which has the rust-toolchain file with X toolchain specified, Cargo will not use X. It will use the default toolchain A is compiled with, also for compiling B. Otherwise, we would have (abi?) conflicts between the two.

The problem here is that Siderophile somehow respects B crate's rust-toolchain file and tries to compile it with X and then we have a clash.

I prepared a [failing] PoC in #22 and I am trying to track down why this occurs and how to fix that. It seems that the compilation of B fails on https://github.com/trailofbits/siderophile/blob/master/src/deps.rs#L434 as in:

➜  crate-use-rust-toolchain git:(add-rust-toolchain) ✗ ../../target/debug/siderophile -o ./siderophile_out/unsafe_deps.txt
    Checking bitvec v0.15.2
error: 'rustc' is not installed for the toolchain '1.39.0-x86_64-apple-darwin': bitvec
To install, run `rustup component add rustc --toolchain 1.39.0-x86_64-apple-darwin`
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Cargo("Could not compile `bitvec`.")', src/libcore/result.rs:1187:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

I have also tried comparing e.g. the rustc command seen with cargo build --verbose:

➜  crate-use-rust-toolchain git:(add-rust-toolchain) ✗ cargo build --verbose
   Compiling bitvec v0.15.2
     Running `rustc --edition=2018 --crate-name bitvec /home/.cargo/registry/src/github.com-1ecc6299db9ec823/bitvec-0.15.2/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C debuginfo=2 --cfg 'feature="alloc"' --cfg 'feature="atomic"' --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=25a11940d5ffd3fb -C extra-filename=-25a11940d5ffd3fb --out-dir /siderophile/tests/crate-use-rust-toolchain/target/debug/deps -L dependency=/siderophile/tests/crate-use-rust-toolchain/target/debug/deps --cap-lints allow`
   Compiling crate-uses-rust-toolchain v0.1.0 (/siderophile/tests/crate-use-rust-toolchain)
     Running `rustc --edition=2018 --crate-name crate_uses_rust_toolchain src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type lib --emit=dep-info,metadata,link -C debuginfo=2 -C metadata=daa7f2c064067706 -C extra-filename=-daa7f2c064067706 --out-dir /siderophile/tests/crate-use-rust-toolchain/target/debug/deps -C incremental=/siderophile/tests/crate-use-rust-toolchain/target/debug/incremental -L dependency=/siderophile/tests/crate-use-rust-toolchain/target/debug/deps --extern bitvec=/siderophile/tests/crate-use-rust-toolchain/target/debug/deps/libbitvec-25a11940d5ffd3fb.rmeta`
    Finished dev [unoptimized + debuginfo] target(s) in 2.52s

With the command executed at src/deps.rs#L434 I was able to capture by adding some debug logs to Siderophile:

CARGO_MANIFEST_DIR="/home/.cargo/registry/src/github.com-1ecc6299db9ec823/bitvec-0.15.2" \
CARGO_PKG_NAME="bitvec" \
CARGO_PKG_DESCRIPTION="A crate for manipulating memory, bit by bit" \
CARGO_PKG_HOMEPAGE="https://myrrlyn.net/crates/bitvec" \
CARGO_PKG_AUTHORS="myrrlyn <self@myrrlyn.dev>" \
CARGO_PKG_REPOSITORY="https://github.com/myrrlyn/bitvec" \
CARGO=/home/.rustup/toolchains/nightly-x86_64-apple-darwin/bin/cargo \
CARGO_PKG_VERSION_MINOR="15" \
CARGO_PKG_VERSION_PRE="" \
CARGO_PKG_VERSION_PATCH="2" \
CARGO_PKG_VERSION_MAJOR="0" \
DYLD_FALLBACK_LIBRARY_PATH="/siderophile/tests/crate-use-rust-toolchain/target/debug/deps:/home/.rustup/toolchains/nightly-x86_64-apple-darwin/lib:/home/lib:/usr/local/lib:/lib:/usr/lib" \
rustc --verbose --edition=2018 --crate-name bitvec /home/.cargo/registry/src/github.com-1ecc6299db9ec823/bitvec-0.15.2/src/lib.rs --color always --crate-type lib --emit=dep-info,metadata -C debuginfo=2 --cfg 'feature="alloc"' --cfg 'feature="atomic"' --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=685d4ed175c41faf -C extra-filename=-685d4ed175c41faf --out-dir /siderophile/tests/crate-use-rust-toolchain/target/debug/deps -L dependency=/siderophile/tests/crate-use-rust-toolchain/target/debug/deps --cap-lints allow

and I can't find what makes this difference. There is e.g. difference in --emit -- the working one (cargo build) adds link there. Also, this command doesn't print anything when run standalone :/.

[disconnect3d]

An obvious workaround is of course explicitly using the RUSTUP_TOOLCHAIN environment variable as specified in this issue:

➜  crate-use-rust-toolchain git:(add-rust-toolchain) ✗ RUSTUP_TOOLCHAIN='nightly-x86_64-apple-darwin' ../../analyze.sh crate-uses-rust-toolchain
trawling source code of dependencies for unsafety
    Checking bitvec v0.15.2
    Checking crate-uses-rust-toolchain v0.1.0 (/Users/dc/tob/projects/siderophile/tests/crate-use-rust-toolchain)
    Finished dev [unoptimized + debuginfo] target(s) in 2.34s
generating LLVM bitcode for the callgraph
   Compiling bitvec v0.15.2
   Compiling crate-uses-rust-toolchain v0.1.0 (/Users/dc/tob/projects/siderophile/tests/crate-use-rust-toolchain)
    Finished dev [unoptimized + debuginfo] target(s) in 2.56s
generating callgraph
WARNING: You're attempting to print out a bitcode file.
This is inadvisable as it may cause display problems. If
you REALLY want to taste LLVM bitcode first-hand, you
can force output with the `-f' option.

Writing 'callgraph.dot'...
unmangling callgraph symbols
matching unsafe deps with callgraph nodes
tracing the unsafety up the tree
done. see siderophile_out/badness.txt

[disconnect3d]

Fixed via a workaround in #22.

I opened #25 that explains future work on that matter.