You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
#1526 added a MemoryManager along with the concept of a RuntimeEnv, this is awesome. Unfortunately ExecutionContext::new() and ExecutionContext::with_config call RuntimeEnv::new(), which in turn creates a new MemoryManager and DiskManager, which create a temporary directory and print logs, which is decidedly less awesome.
At least within IOx we construct an ExecutionContext for each query, as various settings may change, most notably the catalog. Therefore these logs and especially the directory creation and associated IO (#1637) are problematic.
Describe the solution you'd like
I think it should be possible to pass an extant Arc<RuntimeEnv> to an ExecutionContext, so that multiple different ExecutionContext can share the same resource pool.
Describe alternatives you've considered
We could maintain separate ExecutionContext for each type of query IOx executes, but this is just punting the problem imo. We want to be able to have queries running in different contexts, but on the same pool of resources managed by a RuntimeEnv.
Activity
alamb commentedon Jan 21, 2022
Being able to do something like
makes a lot of sense to me
yjshen commentedon Jan 23, 2022
Thanks for the detailed explanation of the use case in Influx, haven't thought of it before. I think it would be great to have this.
alamb commentedon Jan 24, 2022
I plan to work on this issue, with respect to https://github.com/influxdata/influxdb_iox/issues/3507
alamb commentedon Jan 24, 2022
In general I am planning a cleanup to the
MemoryManagerandDiskManagerconfiguration. I expect to have a PR for review shortlyalamb commentedon Jan 24, 2022
PR with proposed improvements: #1668