Arbitrary kevent support #578

Activity

1. njsmith commented on Jul 30, 2018

   

   njsmith

   on Jul 30, 2018

   Member

   More actions

   Okay, I think I've swapped this back into my brain now, and looking at #579...

   First, the thing where monitor_kevent didn't actually register the kevent was intentional – if you look at the comment above current_kqueue, it has some discussion of this. The thing is that the semantics for which kevents you can get when are actually super complicated – like you can request kevent X and then get kevent Y, or you can pass the kqueue to some other syscall entirely and then kevent Z magically appears. So my plan was for us to kind of throw up our hands and tell people who want to use the low-level API that OK, here's your raw kqueue, here's a primitive to listen for kevents, you register for what you want and then listen for what you want and please be very careful not to screw things up because that will break everything, but really it's Python so we can't stop you.

   But, looking at it again, though, I'm not sure this is a great plan. There are two cases I know of where kevent filters can magically get registered without using KV_ADD. These are:

   • AIO: you can implicitly register a kevent filter by passing a kqueue and some flags to aio_read, etc. It's not supported by MacOS, though, so we can't test it. Also, using AIO from Python would take a lot of work (none of the relevant syscalls are wrapped, and you lose the entire Python I/O stack)... not super interesting or urgent. The API I described above can handle this.

   • EVFILT_PROC with NOTE_TRACK: this allows you to not just monitor a process, but also request that if the process creates any child processes, then you want to automatically (and atomically!) register filters for them as well. This isn't supported by MacOS either. It's pretty useful in theory – I can easily imagine a FreeBSD shop wanting to use this (so... basically just Netflix I guess). But the API I described above can't handle this. For NOTE_TRACK, as soon as you see a notification that a child has been created, you have to immediately – like, inside handle_io – add a note to self._registrations to start directing the child's results to the appropriate place. So this needs a special-case to work, and once we have the special case, it no longer needs the API I described above.

   So, never mind all that. Let's drop current_kqueue. Now, what should a basic "add a filter and listen to it and remove it" API look like?

   I think we probably want to public functions: one that listens for a single ONESHOT event, and one that listens for events on an ongoing basis. The first can probably be a simple wrapper around the second (that listens until it hears one event and then returns). For the latter... that's basically what #579 makes monitor_kevent (though we might want a way to pass through EV_ONESHOT? I'm not sure if any of the other flags are useful). So that's pretty reasonable. And wait_kevent could become the EV_ONESHOT method.

   One more wrinkle: how monitor_kevent reports events. Right now it uses a trio.hazmat.UnboundedQueue. Which is workable, and if you want to leave it as that for now then I don't blame you and we can always revisit it later :-). But UnboundedQueue is a holdover from prehistoric trio and it's a bit odd – this (and an equivalent stub method in _io_windows.py) is literally it's only remaining usage, and it's not actually terribly well suited for this. It provides irrelevant functionality, and also is limited in that we can't mark it as closed when the kevent is unregistered (see the NotImplementedError in notify_fd_close for an example of this being an issue).

   So I'm wondering if we should replace the UnboundedQueue with a more specialized object, something like:

   @attr.s
   class KqueueMonitor:
       data = attr.ib(default=attr.Factory(list))
       closed = attr.ib(default=False)
       waiter = attr.ib(default=None)
   
       def _append(self, event):
           self.data.append(event)
           if self.waiter is not None:
               _core.reschedule(self.waiter)
   
       def _close(self):
           self.closed = True
           if self.waiter is not None:
               _core.reschedule(self.waiter)
   
       def __aiter__(self):
           return self
   
       async def __anext__(self):
           if not self.data and not self.closed:
               if self.waiter is not None:
                   raise ResourceBusyError
               task = _core.current_task()
               self.waiter = task
               def abort_fn(_):
                   self.waiter = None
                   return _core.Abort.SUCCEEDED
               await _core.wait_task_rescheduled(abort_fn)
           if self.data:
               data = self.data
               self.data = []
               return self.data
           assert self.closed
           raise ClosedResourceError
2. oremanj

   added

   low-level

   on May 4, 2019
3. arthur-tacca commented on Sep 17, 2026

   

   arthur-tacca

   on Sep 17, 2026

   Contributor

   More actions

   For subscribers to this issue: there is related discussion #3500 (later comments) and #2922.
4. arthur-tacca

   added a commit that references this issue on Oct 1, 2026

   wait_kevent: do the kevent registration itself (part of python-trio#578)

   ...

   9455852