Skip to content

filename is encoded to utf8, resulting in broken filenames on windows #686

Closed

Description

@MaxBo

When i try to create a netcdf-file with a filepath, that contains Non-ASCII-characters, Netcdf4 creates broken filenames on windows, because it encodes the filepath in UTF8, which is fine for Linux,. In Windows, the filepath has to be convertet to CP1252.

import netCDF4 as nc4
filepath = r'C:\Plön.nc'
ds = nc4.Dataset(filepath, 'w')
ds.close()

this creates a file
Plön.nc

I could try to avoid Non-ASCII-Characters in .nc filenames, but if i want to create a file in the folder
C:\Plön\

import netCDF4 as nc4
filepath = r'C:\Plön\Ploen.nc'
ds = nc4.Dataset(filepath, 'w')
ds.close()

then i get the confusing error message:
builtins.OSError: Permission denied
Because the folder "Plön" is not recognized correctly.

The solution would be to pass the filesystemencoding to _strencode in
_netCDF4.pyx in line 1796

-  bytestr = _strencode(str(filename))
+ bytestr = _strencode(str(filename), encoding=sys.getfilesystemencoding())

I'll create a pull request.
Max

Activity

  1. added a commit that references this issue on Jul 3, 2017
    0878ea3
  2. jswhit commented on Jul 5, 2017

    @jswhit
    Collaborator

    Reopening this - tst_filepath.py is failing on systems with default encoding of 'utf-8'. The only reason the travis tests passed is that this test was skipped since the version of the netcdf library is so old.

    ======================================================================
    ERROR: test_filepath_with_non_ascii_characters (__main__.test_filepath)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "tst_filepath.py", line 23, in test_filepath_with_non_ascii_characters
        assert nc_non_ascii.filepath() == str(nc_non_ascii_file)
      File "netCDF4/_netCDF4.pyx", line 1953, in netCDF4._netCDF4.Dataset.filepath 
    (netCDF4/_netCDF4.c:15031)
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 49: ordinal not in range(128)
    

  3. added a commit that references this issue on Jul 5, 2017
    737f97e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions