Better ergonomics for working with MultiErrors #49

Activity

1. njsmith

   added

   design discussion

   user happiness

   on Feb 12, 2017
2. njsmith commented on Feb 15, 2017

   

   njsmith

   on Feb 15, 2017

   MemberAuthor

   More actions

   Idea for catching/filtering exceptions when there are MultiErrors around:

   def exception_filter():
       try:
           yield
       except ...:
           pass
       except ...:
           raise
   
   with MultiError.filter(exception_filter):
        ...

   Where the semantics are: the __exit__ block effectively maps exception_filter over all the exceptions that were raised (if any).

   This will probably need better logic for flattening MultiError (or at least recursing over the tree), now that I think about it... in fact I think cancel_scope might get this wrong :-(
3. njsmith commented on Feb 15, 2017

   

   njsmith

   on Feb 15, 2017

   MemberAuthor

   More actions

   Another weirdness: if an exception gets packed into a MultiError, and then unpacked again later, then part of its traceback just disappears.

   In general the packing and nesting is kind of nice because it means shared branches of the stack only get saved once, instead of being repeated in all of the atomic exceptions. But when we revert to the atomic exceptions it's kinda sucky :-(.

   ...Maybe there's some horrible way to concatenate the traceback objects?
4. njsmith commented on Feb 15, 2017

   

   njsmith

   on Feb 15, 2017

   Last edited by njsmith

   MemberAuthor

   More actions

   It looks like traceback objects are linked lists with tb_next pointing to the next item. So concatenating them is a simple matter of assigning to tb_next. Unfortunately tb_next is readonly :-(. Of course nothing is really readonly in CPython... though PyPy might be another story. [Edit: the hacky solution for PyPy is transparent proxies. Contra to what that page says, they appear to be enabled by default.]

   Also, it's not enough to concatenate them -- we need to be able to copy them, because when reconstructing a MultiError after a multi-catch, the original single MultiError's traceback might need to be concatenated to multiple exceptions.

   [Edit: it looks like jinja2 has an implementation of a good chunk of what we need.]
5. njsmith commented on Feb 16, 2017

   

   njsmith

   on Feb 16, 2017

   Last edited by njsmith

   MemberAuthor

   More actions

   I also wonder if we should go ahead and override sys.excepthook (if not already overridden) to something that handles MultiErrors appropriately? It's unfortunate that this won't help in ipython or other enhanced interpreters, but... (I guess for ipython there's always the monkeypatch approach!) [Edit: oh, here's a real solution for ipython]
6. njsmith commented on Feb 18, 2017

   

   njsmith

   on Feb 18, 2017

   MemberAuthor

   More actions

   If this python issue were fixed somehow it would be helpful...: https://bugs.python.org/issue18861
7. njsmith commented on Feb 18, 2017

   

   njsmith

   on Feb 18, 2017

   Last edited by njsmith

   MemberAuthor

   More actions

   More notes on ipython custom tracebacks:

   There can only get one custom exc handler registered. So we should so something like

   ip = IPython.get_ipython()
   if ip.custom_exceptions != ():
       warnings.warn("IPython detected, but you already have a custom traceback handler installed ...")
   else:
       ip.set_custom_exc(...)

   We can't return the traceback info to be formatted (I think?), but instead must print it ourselves, because we can only return one traceback (as a list of strings). But tracebacks eventually end up in ip._showtraceback(etype, value, list_of_strings), which we can call if we want.

   The way IPython generates the list-of-strings is with ip.InteractiveTB.structured_traceback(etype, value, tb, tb_offset=tb_offset). However, we can't directly call this repeatedly, for two reasons:

   • It's stateful – it saves the traceback (maybe for %pdb? not sure). Maybe harmless, maybe bad.

   • It has baked into it that it should discard the top two frames of the traceback that's passed in (though you can have it discard more via tb_offset, it seems to use max(tb_offset, 2). This makes sense for stuff caught at the IPython top-level, but not for intermediate exceptions.
8. njsmith commented on Feb 18, 2017

   

   njsmith

   on Feb 18, 2017

   MemberAuthor

   More actions

   Better tracebacks in pytest would also be really nice. This looks noticeably harder, though. I think it might be possible with a custom collector that defines a custom repr_failure method?
9. njsmith commented on Feb 20, 2017

   

   njsmith

   on Feb 20, 2017

   MemberAuthor

   More actions

   This is a messy thread and the main issues are fixed; closing in favor of #27 and #56
10. njsmith

    closed this as completedon Feb 20, 2017