Repository navigation
speedup of pluggy #322
Description
there is multiple levels of speedups possible/for consideration
- external tools (mypyc/cython)
- the hook call mechanism (attr getter in c for the arguments)
- setup/teardown of hooks
- behaviour of the limited/scoped hook callers (subset callers used in pytest)
to better gauge impacts we need benchmarks stressing
- nesting of calls
- number of calls
- number of plugins
- surrounding tools like tracing/hook exclusion
- number of arguments
Metadata
Metadata
Assignees
Labels
No labels
Activity
wahuneke commentedon Sep 26, 2023
Would you be interested in seeing a POC version using a 'dynamic code' approach? The concept here would be: at the first attempt to run hooks through HookRelay, we 'precompile' the whole stack of hooks so that less logic needs to be evaluated during the hook call and subsequent calls. Modifying the stack (by (de)registering plugins, etc) would invalidate the cached 'compiled' version.
By preprocessing / 'compiling' the stack we accelerate some of the slower aspects of calling hooks, specifically:
In my local, rough draft implementation, I was able to speedup benchmarks by 10-20% with only a small modification to the current code. More speed improvement is probably possible under this approach.
In some ways, I think, generated code is a great fit for what pluggy wants to do. On the other hand, I'm not sure it is ever a good idea to use it in a real project :P . Any thoughts?
RonnyPfannschmidt commentedon Sep 26, 2023
It's not quite clear what you mean by that,
Cython is currently Most likely to create a spedup
Certain refactoring need to be added first tho
wahuneke commentedon Sep 27, 2023
Sorry @RonnyPfannschmidt ! That was a terrible explanation. I've taken one more run at it. And, I've moved the discussion to a separate ticket in order to avoid polluting this one (issue #449)