Summary
In our CI, we use jemalloc for all targets except Windows. Previously this was implemented by setting rust.jemalloc = true globally, then overriding for Windows targets with target.x86_64-pc-windows-msvc.jemalloc = false
Unfortunately this no longer works since #155617 was merged.
Command used
Bootstrap.toml: see below
Add tracing of inputs and outputs to reconcile_jemalloc and Config::override_allocator in src/bootstrap/src/core/config/config.rs
Then run basically any command, eg. ./x test bootstrap
Expected behaviour
The target-specific directive should take precedence, so the system allocator should be used
Actual behaviour
(reproduced on a Mac, but the equivalent should happen on any platform if you change the directive in bootstrap.toml accordingly)
reconcile_jemalloc: section="target.aarch64-apple-darwin", jemalloc = Some(false), override_allocator = None, result = None
reconcile_jemalloc: section="rust", jemalloc = Some(true), override_allocator = None, result = Some(Jemalloc)
Config::override_allocator: target=aarch64-apple-darwin, rust.override_allocator=Some(Jemalloc), target.override_allocator=None, result=Some(Jemalloc)
Config::override_allocator: target=aarch64-apple-darwin, rust.override_allocator=Some(Jemalloc), target.override_allocator=None, result=Some(Jemalloc)
ie. the explicit jemalloc = false directive is being remapped to override_allocator being unset (None as opposed to Some(value)). So Config::override_allocator falls back to the global directive.
On Mac the build succeeds from here (but with an unintended configuration) because jemalloc works; the equivalent on Windows fails to build because jemalloc is not supported there.
Bootstrap configuration (bootstrap.toml)
profile = "compiler"
change-id = "ignore"
[rust]
jemalloc = true
[target]
aarch64-apple-darwin.jemalloc = false
Operating system
All
HEAD
e7b5955
Additional context
I believe the proper resolution here is to add a "system" option to override-allocator, so that we can disambiguate "target.[...].override-allocator is not set" from "target.[...].override-allocator explicitly says to not override the allocator on this platform"
Then jemalloc = false should be mapped to override-allocator = "system" instead of being mapped to override-allocator being unset
I plan to put together a fix and submit it soon.
Summary
In our CI, we use jemalloc for all targets except Windows. Previously this was implemented by setting
rust.jemalloc = trueglobally, then overriding for Windows targets withtarget.x86_64-pc-windows-msvc.jemalloc = falseUnfortunately this no longer works since #155617 was merged.
Command used
Bootstrap.toml: see below
Add tracing of inputs and outputs to
reconcile_jemallocandConfig::override_allocatorinsrc/bootstrap/src/core/config/config.rsThen run basically any command, eg.
./x test bootstrapExpected behaviour
The target-specific directive should take precedence, so the system allocator should be used
Actual behaviour
(reproduced on a Mac, but the equivalent should happen on any platform if you change the directive in bootstrap.toml accordingly)
ie. the explicit
jemalloc = falsedirective is being remapped tooverride_allocatorbeing unset (Noneas opposed toSome(value)). SoConfig::override_allocatorfalls back to the global directive.On Mac the build succeeds from here (but with an unintended configuration) because jemalloc works; the equivalent on Windows fails to build because jemalloc is not supported there.
Bootstrap configuration (bootstrap.toml)
Operating system
All
HEAD
e7b5955
Additional context
I believe the proper resolution here is to add a "system" option to
override-allocator, so that we can disambiguate "target.[...].override-allocator is not set" from "target.[...].override-allocator explicitly says to not override the allocator on this platform"Then
jemalloc = falseshould be mapped tooverride-allocator = "system"instead of being mapped to override-allocator being unsetI plan to put together a fix and submit it soon.