parse_wpc_surface_bulletin does not handle StringIO object, results in AttributeError: 'str' object has no attribute 'decode' #3923

Activity

1. akrherz

   added

   Type: BugSomething is not working like it shouldSomething is not working like it should

   on Sep 26, 2025
2. dopplershift commented on Sep 26, 2025

   

   dopplershift

   on Sep 26, 2025

   Member

   More actions

   @akrherz This should work fine if you replace StringIO with BytesIO. I'm not sure how we can detect whether the decode() call is needed without some ugly type-checking for what read() returns.
3. DWesl commented on May 9, 2026

   

   DWesl

   on May 9, 2026

   Contributor

   More actions

   Checking whether decode is possible/succeeds would be simpler:

   text = file.read()
   try:
       text = text.decode('utf-8')
   except AttributeError:
       pass
4. dopplershift commented on May 11, 2026

   

   dopplershift

   on May 11, 2026

   Member

   More actions

   Given that text comes from a call to read(), this isn't so bad. At the same time, I hesitate because this is essentially undoing/revisiting what was an important Python 2->3 change; I just wonder if we're opening a door to a bunch of complicated problems.
5. DWesl commented on May 12, 2026

   

   DWesl

   on May 12, 2026

   Last edited by DWesl

   Contributor

   More actions

   In that case, you'll want to document that parse_wpc_surface_bulletin only accepts ASCII- or UTF-8-encoded binary-mode file-like objects, not text-mode file-like objects nor binary-mode file-like objects with different encodings, so they don't try

   with open(name, mode="r", encoding="utf-16") as in_file:
       synoptic_setup = parse_wpc_surface_bulletin(in_file)

   but instead

   with open(name, mode="r", encoding="utf-16") as in_file:
       synoptic_setup = parse_wpc_surface_bulletin(io.BytesIO(in_file.read().endode("utf-8")))

   or run the file through iconv before attempting to open it

   (or even

   with open(name, mode="r", encoding="us-ascii") as in_file:
       synoptic_setup = parse_wpc_surface_bulletin(in_file)

   )

   You may also want to make a general statement on the metpy.io page that all files are expected to be passed in binary mode, and therefore in-memory examples should use io.BytesIO rather than io.StringIO, to explain why a file with requirements for its text encoding is expected as a binary-mode file rather than a text-mode file

   (Alternately, add an encoding argument to parse_wpc_surface_bulletin and open_if_needed, defaulting to UTF-8 in the former and None in the latter, since you expect the file to contain text)