Must static DATA: UnsafeCell = expr; still == expr by main()?

@RalfJung

"Okay I think we first and foremost have a Rust Abstract Machine / @rust-lang/opsem question here, not a Miri question:

If a mutable (or interior mutable) static is initialized in Rust to a certain value, then can the Rust AM assume that it does have that value when the program starts? Or is it okay to have some "before main setup" actually change the value of that static?

I would say for an immutable static, this is clearly UB -- using linker script tricks like what has been described here is just not allowed for those statics. But for statics that can be mutated, we already can't in general optimize assuming their values did not change, so... it seems reasonable to allow this?

With this way of thinking about the question, obviously Miri has no way of knowing that you are modifying the value of the static before Rust code starts running. On the Rust side you are saying this static is filled with uninit memory, but then you are making some outside-of-Rust assumptions to justify that actually when the program starts the static has a different value, in particular that the bytes now are initialized. So I don't think there is anything actionable on the Miri side here, and the discussion (with the clarified question) should move to https://github.com/rust-lang/unsafe-code-guidelines/ ."

Originally posted by @RalfJung in rust-lang/miri#2807 (comment)

I think that mutable statics still need to have the values they were promised to have in the initializer unless an AM-visible write occurred to them. That write might have been caused by FFI or asm!, but if the value isn't there at lang_start then it's not clear where the value goes at all, in which case a compiler which just ignores all static mut initializers would be conforming and that's not ok.

Regarding the linked thread, I think the appropriate model is to use an extern static and allocate/uninitialize it using a linker script, and then use as the AM invariant that when main() is called that static is initialized with a random bit pattern (not uninit). That's still not quite enough to validate the technique of signature validation since that's only probabilistically correct in the first place, but it does eliminate the need to do any volatile reads and writes and you won't get UB.

I don't see a way to make this work with a rust-allocated static without lying to the compiler to some extent. For example, we could use a mutable static (initialized to 0, say) with a link_section = ".uninit" on it, and then call an empty asm! block which has the postcondition of ensuring that the static is initialized to a random bit pattern, but in that case the AM will be in disagreement with the hardware between the start of main() and the asm! block, since the AM thinks the static is zero-initialized and the hardware static has actual data in it. (This is "fine" since the static will not be read or written to during this period.)

@digama0

What if you did this:

use std::arch::asm;

static mut FOO: i32 = 0;

pub fn main() {
    unsafe {
        asm!("/* {} */", in(reg) &mut FOO);
    }
    // Rest of program...
}

The ASM block does nothing but the compiler must behave as though it may have written a value into the static. I guess this is what you were suggesting @digama0 ?

Yes, that is what I meant. The drawback is that the compiler is technically allowed to read FOO before the asm! block and assume that the value in the static is 0 because no threads have started yet so things should still look like the initialization values. In practice this doesn't seem to be a likely issue, since there isn't much optimization reason to do so.

(Also, we should replace main() with lang_start() in this discussion; there is some std code which runs before main which is not special in this regard. This should be fine for the present discussion since it certainly doesn't touch FOO.)

It's not completely unreasonable to have a step between codegen where the initial starting parameters of the AM are serialized and the start entry point into AM execution where the host could be allowed to insert some set of AM state changes, which could include changing the value of static muts.

Notably, the host can expose some known memory locations' provenance, such that from_exposed_addr(MMIO_ADDRESS) can pick up valid provenance; I think that being valid is fairly non-controversial. If the startup state can include exposed addresses from the host in the AM ambient state, I find it more difficult to say why other host-writable AM state cannot be modified between codegen and startup.

But I still agree that the proper way to model the original use case here is almost certainly an extern static allocated by the host linker shenanigans rather than managed within the AM.

I think for as long as DATA is exported (using eg #[no_mangle]), it should be possible to write to it before main. I don't think the main function should be special cased. There is no main function for cdylib's and for cdylib's I don't see any reason to not allow writing to DATA from the user of the cdylib before the first time calling an exported function. I think a binary should be treated just like a cdylib except always exporting the main shim which calls lang_start and the user main function. If on the other hand DATA is not exported outside of the rust crate, I think it is fair to require that it has the exact same value as the initializer upon the first call to an exported function and does not get changed outside of the control of functions in the rust crate. So for example if there are no functions that modify it, ot could be moved fo read-only memory.

I think for as long as DATA is exported (using eg #[no_mangle]), it should be possible to write to it before main. I don't think the main function should be special cased.

I don't think this is about special casing, so much as the issue that there is no "life before main", even in the AM. As far as I am aware the AM semantics start at lang_start(). So FFI which calls lang_start() is another level of subtle compared to regular FFI.

As I mentioned before, the issue with saying that the initialization value of a static doesn't need to be in that static when the AM takes its first step (wherever we decide to put that) is that it becomes impossible for the AM to distinguish between static mut FOO: u32 = 0; and static mut FOO: u32 = 1; as declarations - these are observably equivalent and a compiler could just ignore the initializers if it wanted to. This is obviously not what we want.

The way the analogy breaks down with cdylibs is that a call to a rust function from FFI is not the AM's first step. There is still some subtlety to define what exactly was happening in the AM before the first line of rust code, but I imagine that it would be something like "arbitrary RAM-legal operations, such that the state at the call satisfies the appropriate calling convention". Since it's an unsafe interface we can freely add additional safety constraints like "and FOO should have value 0" to control the state of the machine when the function is called; for lang_start it is unclear how we could do anything like that because it's a safe function.

@CAD97

I think that mutable statics still need to have the values they were promised to have in the initializer unless an AM-visible write occurred to them. That write might have been caused by FFI or asm!, but if the value isn't there at lang_start then it's not clear where the value goes at all, in which case a compiler which just ignores all static mut initializers would be conforming and that's not ok.

No that's not correct. Just because the user is allowed to change the value via arbitrary means before lang_start, doesn't mean the compiler is allowed to do that.

For the same reason your "observably equivalent" argument doesn't hold up. This issue is not about giving the compiler more choice, it is purely about giving the user more choice. @CAD97 expressed it pretty well I think, it's basically a user-decided machine step that happens before the lang_start stack frame is pushed.

@CAD97

For the same reason your "observably equivalent" argument doesn't hold up. This issue is not about giving the compiler more choice, it is purely about giving the user more choice. @CAD97 expressed it pretty well I think, it's basically a user-decided machine step that happens before the lang_start stack frame is pushed.

I mean, I'm fine with building the equivalent of an empty asm! block into the language semantics for the start of the machine, but that still leaves me confused as to how this actually affects the spec. What exactly is the compiler promising when we compile a regular rust program with a static mut and no linker shenanigans? How precisely are you going to prevent a compiler that ignores initializers from being counted as conforming?

A completely user-decided machine step at program start is also a really heavy hammer. If you aren't careful, you might make it impossible for the compiler to produce a working program under those conditions since the compiler doesn't know what the user intends there.

I'm confused about why you are bringing up this question since it doesn't change with this issue at all? In the initial machine state (before that new user-defined step we are discussing) the static mut has the value given by the initializer.

But I still agree that the proper way to model the original use case here is almost certainly an extern static allocated by the host linker shenanigans rather than managed within the AM.

That seems to be hard to do without a C toolchain, or something? I don't claim to understand the usecase that triggered the discussion here.^^

I'm confused about why you are bringing up this question since it doesn't change with this issue at all? In the initial machine state (before that new user-defined step we are discussing) the static mut has the value given by the initializer.

This user-defined step probably needs quite some work to unpack, since it is basically what we have been calling "linker shenanigans". Somehow the spec has to interact with linker scripts, and I guess it is this code that would actually see the values of static initializers and have an opportunity to change them.

As it relates to the title question, I think the answer should still be "yes" provided there are no linker scripts (or only the default one). A program like:

static mut FOO: u8 = 0;
fn main() {
  println!("{}", unsafe { FOO });
}

with no special linking stuff should be required to print 0 (and the user should not be able to get this to print 1 by wishing on a star). Anything which changes the output of this program needs to involve changing an actual input to rustc (+ linker + dynamic linker).

But I still agree that the proper way to model the original use case here is almost certainly an extern static allocated by the host linker shenanigans rather than managed within the AM.

That seems to be hard to do without a C toolchain, or something? I don't claim to understand the usecase that triggered the discussion here.^^

If true, this seems like a language issue. It should be possible to declare "extern statics" with link_section using only rust and a linker script (not that I know much about the latter). I think the OP was talking about using C to allocate the static itself, which would be just as incorrect as it is in rust.

A program like:
static mut FOO: u8 = 0;
fn main() {
  println!("{}", unsafe { FOO });
}
with no special linking stuff should be required to print 0 (and the user should not be able to get this to print 1 by wishing on a star).

That is satisfied by my suggestion in #397 (comment) to require that only exported statics can be mutated by external processes. If you were to add pub or #[no_mangle] I think it should be allowed to print something else, but without both I agree that it should only print 0.

Ah, sorry I should have made it pub. I am talking specifically about the case where it is public but nothing is linked in which could actually do any funny business. In such a case, external processes are allowed to mutate it, but this is not the same as them having already done so. The AM sees all, and unless such a process actually runs no mutation should happen.

In case of pub for bin's specifically I can agree. But in case of #[no_mangle] you can't prove that the libc that is linked in the above program doesn't do dlsym to lookup FOO and mutate it.

I agree, but that's more of a compiler optimization perspective than what the AM actually does. The compiler doesn't know what the dynamic linker is doing either, so it can only do e.g. constant folding if it knows or has an agreement with all such code that the relevant statics will not be touched, but the AM can say more precisely "The program prints 0 as long as libc et al don't mutate FOO".

We absolutely have a linker script (what embedded SW doesn't? :) ). But the linker doesn't do allocation, it just assigns sections to memory. The sections are coming from the object files though. But perhaps this is the problem (because you're correct that I can do "allocation" in the linker script). I think with Rust extern static that would be required, but I still need to run those experiments.

The data absolutely is there before Rust starts. In the same way that .data gets populated with initialized non-zero data and .bss gets zero'd out before the first line of Rust code starts, this particular memory is also populated. The only difference compared to .data and .bss is that Rust (when it starts) does NOT know what is in that memory, whereas it does know what is in .bss (it is all zeros) and .data (it matches the explicit initialization declared in the Rust code). My worry is that Rust will think it knows and do something with what it thinks.

But the linker doesn't do allocation, it just assigns sections to memory.

I think you can do something like

.persistent : {
    my_global = .;
    . += 1024; /* assign 1024 bytes to the .persistent section. */
} > RAM

with 1024 replaced by whatever size you want.

Rust generates an object file instructing the linker to fill a particular region of memory with 0s. But then actually you set things up in a way that the memory contains other data. Is that correct?

That does seem questionable, though maybe one can argue that it is equivalent to first having it initialized with 0s and then having some outside-of-Rust code run that writes the data you actually want to see there. So as long as the static is (interior) mutable, this will probably work... but I would argue extern static + linker script is a more robust solution.

But the linker doesn't do allocation, it just assigns sections to memory.

I think you can do something like
.persistent : {
    my_global = .;
    . += 1024; /* assign 1024 bytes to the .persistent section. */
} > RAM
with 1024 replaced by whatever size you want.

Yes this would be the kind of thing one would do. Unfortunately this means the size info now has to be declared in two distinct places (the linker file and the Rust code file). For more complicated composite types this could get very annoying. For simple byte arrays, still not ideal but not a show-stopper. This is why it is much preferred to have the allocation, via declaration and compilation, happen in code. And this is typically what embedded C guys do.

And this is typically what embedded C guys do.

If we can get a documented guarantee from LLVM that this kind of thing is sound to do, it should be fairly easy to also provide this guarantee on the Rust level.

Some godbolt output for 32-bit ARM (common embedded target):

C version: https://godbolt.org/z/rvxr86v7z
Rust pub static mut with init: https://godbolt.org/z/vorj3cYsd
Extern static Rust: https://godbolt.org/z/414MTfx1z

So the allocation doesn't happen in the last case, as predicted. The Rust warning in that case is also a bit interesting (pub static mut a: [u8; 1024]; is "not a function or static" - OK, if you say so.)

Well I ended up going with something like this: https://godbolt.org/z/5qjaesz9z
The explicit initializer is MaybeUninit::uninit() rather than something like [0u8; SIZE]. Then instead of doing write() on that, followed by assume_init(), we basically just skip the write and only do the assume_init using assume_init_ref(). I feel better about this since I'm not giving Rust any false info about the underlying initialized data. In our real use case we wrap this all up in a SyncUnsafeCell to avoid the static mut complaints.

I looked at the MIR output for the two cases (explicit init vs MaybeUninit::uninit) and the alloc statements for the arrays do show the init vs uninit difference. This seems as close as I can get to the C case where the array is declared global without an initializer.

The explicit initializer is MaybeUninit::uninit() rather than something like [0u8; SIZE].

You could use an extern static right? Then you wouldn't need to provide an initializer, and this issue doesn't affect you.

The explicit initializer is MaybeUninit::uninit() rather than something like [0u8; SIZE].

You could use an extern static right? Then you wouldn't need to provide an initializer, and this issue doesn't affect you.

As mentioned previously extern static is actually not preferred since it forces the "allocation", such as it is, to be done in the linker file (or somewhere else) since the extern static just declares the symbol and it's type. Then your extern static declaration also has to have the size. If it is a simple array, that means you have to have the size in both places. If it is a composite type, then compiler knows the size but the linker does not (so you just have to make sure it is big enough to hold that type). It is actually much preferred that allocation, in the sense of occupying space in the object file, happens in code and only placement in the memory map takes place in the linker (via linker command file content). Even leaving aside the idea of pre-initialization that Rust doesn't know anything about, I could imagine more complicated scenarios with more global objects assigned to the same memory sections, and then alignment (beyond just size) of those object can come into play. Frankly, the codegen tool (rustc) is much better equipped to deal with this problem. I think extern static is really not appropriate. That mainly seems like it would be useful for FFI.

Well, you are doing FFI. Some other component is filling that static with data -- clearly an FFI usecase.

The explicit initializer is MaybeUninit::uninit() rather than something like [0u8; SIZE]. Then instead of doing write() on that, followed by assume_init(), we basically just skip the write and only do the assume_init using assume_init_ref(). I feel better about this since I'm not giving Rust any false info about the underlying initialized data. In our real use case we wrap this all up in a SyncUnsafeCell to avoid the static mut complaints.

FWIW for rustc it makes little difference whether the static is declared with "uninit" and some other component pre-main fills this with data, or whether the static is declared with zeroes and some other component pre-main fills this with data.

extern static cannot possibly be on-topic in this issue, since it is all about what the compiler can deduce based on the initializer expression, and extern static do not have an initializer. So you replied to an off-topic comment (or rather, a part of a comment that discussed how to avoid the question here) and therefore dragged the discussion further off-topic. I'd ask everyone to please stop doing that. :)

That previous sub-thread you mention has trailed off without a resolution since its author disappeared without further clarifying their last comment, which raised more questions than it answered. Also, even the original question was already a misunderstanding of this issue, as indicated by the following comments and questions that never got resolved. I would mark the entire thing as off-topic if Github had anything even close to usable moderation tools, but alas, Github is a terrible venue for discussions of this sort. :(

This is strictly about what the compiler can deduce based on the initializer expression. Github makes is very hard to keep discussions on-topic, please help us by not making it even harder. Just open a new issue for a new question, instead of posting the question in a random existing issue that has some of the right keywords.

Fair enough! My apologies.

Navigation Menu

Repository navigation

Must `static DATA: UnsafeCell = expr;` still `== expr` by `main()`? #397

43 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Must static DATA: UnsafeCell = expr; still == expr by main()? #397

Description

Activity

digama0 commented on Mar 14, 2023

Diggsey commented on Mar 15, 2023

digama0 commented on Mar 15, 2023

CAD97 commented on Mar 15, 2023

bjorn3 commented on Mar 15, 2023

digama0 commented on Mar 15, 2023

RalfJung commented on Mar 15, 2023

digama0 commented on Mar 15, 2023

RalfJung commented on Mar 15, 2023

digama0 commented on Mar 15, 2023

digama0 commented on Mar 15, 2023

bjorn3 commented on Mar 15, 2023

digama0 commented on Mar 15, 2023

bjorn3 commented on Mar 15, 2023

digama0 commented on Mar 15, 2023

43 remaining items

moridinga commented on Jan 28, 2025

bjorn3 commented on Jan 28, 2025

RalfJung commented on Jan 28, 2025

moridinga commented on Jan 28, 2025

RalfJung commented on Jan 28, 2025

moridinga commented on Jan 28, 2025

moridinga commented on Jan 30, 2025

RustyYato commented on Jan 30, 2025

moridinga commented on Jan 30, 2025

RalfJung commented on Jan 31, 2025

dancrossnyc commented on Jun 24, 2025

RalfJung commented on Jun 25, 2025

dancrossnyc commented on Jun 26, 2025

RalfJung commented on Jun 26, 2025

dancrossnyc commented on Jun 26, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Must `static DATA: UnsafeCell = expr;` still `== expr` by `main()`? #397