Repository navigation
Are custom loaders run on their own realm always? #384
Description
In the "How do loaders declare what hooks they implement?" part of the loaders design document, it makes several claims regarding loaders imported by third parties:
Loaders can expose and exported constructor that creates an object with functions matching the names the hooks they implement.
- This allows Loaders to function even within Realms that have frozen globals.
- Globals would not work if Loader has a frozen realm.
- This avoids potential overriding by dependencies by 3rd party modules.
- This guarantees that 3rd party modules could import the hook if they know the specifier of the loader.
- Use policies or similar to restrict this capability.
- This does not allow them to use the same instance of loader object as the runtime since an independent call only made by the runtime can be used to obtain a unique object.
Two claims stick out to me in this mix:
- This avoids potential overriding by dependencies by 3rd party modules.
- This does not allow them to use the same instance of loader object as the runtime since an independent call only made by the runtime can be used to obtain a unique object.
Suppose we have three loaders root_app/loaders/alice.mjs, root_app/loaders/bob.mjs, and node_modules/dep/eve.mjs, each loaded in that order.
// root_app/loaders/alice.mjs
export default class Alice {
// ...
}
// root_app/loaders/bob.mjs
import Alice from "./alice.mjs"
export default class Bob {
// ...
}
// node_modules/dep/eve.mjs
import Alice from "../../loaders/alice.mjs"
let loaders = new Set()
let prev = Alice.prototype.resolve
Alice.prototype.resolve = function (...args) {
loaders.add(this)
return Reflect.apply(prev, this, args)
}
export default class Eve {
// ...
}What would eve.mjs see here? If eve.mjs is in the same realm as the alice.mjs and bob.mjs loaders, or at least share the same module cache, both of the claims in question are false. And based on my reading, it's unclear whether this is the case or not, and I can't find any language explicitly stating one way or the other.
Activity
hybrist commentedon Sep 16, 2019
I would expect them to not share the same loader cache if they're supposed to be isolated. But I agree that it should be called out either way. In general I think module caches should be per realm but that may be beyond this discussion.
devsnek commentedon Sep 16, 2019
loader hooks don't have caches, they respond to the main loader needing to fill in its own cache.
hybrist commentedon Sep 16, 2019
@devsnek If they try to use import/require/etc. would it then fail? Or do you mean "they share the cache" by "they don't have caches"?
devsnek commentedon Sep 16, 2019
@jkrems i'm not sure what you mean by "try to use import/require/etc". each realm has a loader. each loader can have hooks, which run in separate realms/threads.
hybrist commentedon Sep 16, 2019
The loader hooks are implemented in files. Those files will hopefully be able to run as a module. So
importmay be valid syntax inside of the hook implementation. But what does it do? See the original post in this thread for why sharing a require or ESM cache between the loader and the targeted realm is problematic.Something needs to happen when we encounter
requireorimportwithin the loader hook code. Right now (according to @isiahmeadows's investigation) we don't define what happens.devsnek commentedon Sep 16, 2019
ahhhhh i understand now. those imports would be subject to the hooks of the current realm, of which there are none. whether or not we modify the hook behaviour there, the caches of each realm will never be shared. v8 can't link modules from different realms.
hybrist commentedon Sep 16, 2019
Okay, so we agree I think. Each realm/context should have its own module map. And each hook should be considered running in its own context, with its own isolated module map (and require cache). We should call that out in our design docs.
Jamesernator commentedon Sep 17, 2019
If loaders live in their own realm isn't this a bit of a hazard with
dynamicInstantiate?e.g.:
devsnek commentedon Sep 17, 2019
the larger hazard is that we want to run loaders in their own threads, so that would be impossible anyway. i was talking about this with bradley at some point but i don't remember where that conversation ended...
hybrist commentedon Sep 17, 2019
SMotaal commentedon Sep 17, 2019
@jkrems I think it is fair to question the current form of
dynamicInstantiate.Worth noting though that in order to deal with the prototypical realities of the language when aligning loader hooks with forms of dynamic namespace mutations of more popular demand, similar dynamic instantiation mechanisms will be inevitable.
Those hooks apply directly on the instantiated namespace of a module which is bound to specific context (or root realm) primordials… and so this entails that (regardless of technical detail) not all hooks will apply equally… some are pre-cache, some are post, and from those some will likely need to be in realm and operating on respective namespace object instances, where some even operate on a compartment level — imho at least.