SIGSEGV in bundled libhdf5 when reopening a file after closing a handle that read a vlen string variable (1.7.4, 1.7.5.1.post1) #1522

Summary

With the netCDF4 wheels 1.7.4 and 1.7.5.1.post1, opening a netCDF-4 file
crashes the process with SIGSEGV after this sequence in one process:

1. open handle A on the file and keep it open;
2. open handle B on the same file;
3. through B, read a variable-length string variable whose HDF5 fill
   value is a string and whose _FillValue attribute is a vlen string;
4. close B;
5. open the file again: SIGSEGV during netCDF4.Dataset.__init__.

Other orders tested do not crash, for example: B reads and is not
closed; A reads instead of B; A is closed instead of B; no handle is
left open at step 5. Only the netCDF4 extension is loaded in the
crashing process (h5py is used only to write the file, in another
process).

We met it with Sentinel-5P Level-2 products (variable
PRODUCT/time_utc), opened through xarray's netcdf4 engine (which
closes and reopens its handle) while another netCDF4.Dataset was open
on the same file.

Environment

Common: Linux x86_64 (kernel 7.0, glibc 2.43), CPython 3.12.13,
numpy 2.5.4, PyPI manylinux wheels. Writer of the test file:
h5py 3.16.0 (HDF5 2.0.0).

Reproducer (9 kB file)

"""Reproducer: SIGSEGV in libhdf5 when reopening a netCDF-4 file."""
import os
import subprocess
import sys
import tempfile

import h5py
import numpy

path = os.path.join(tempfile.mkdtemp(), 'vlen.nc')
text = h5py.string_dtype('utf-8')
with h5py.File(path, 'w') as f:
    var = f.create_group('PRODUCT').create_dataset(
        'time_utc', shape=(1, 4), dtype=text, fillvalue=b'')
    var[0, :] = [f'2021-03-24T11:30:2{i}Z' for i in range(4)]
    var.attrs.create('_FillValue', numpy.array([''], dtype=object),
                     dtype=text)

# Run in a child process so that the crash shows as a return code.
child = f'''
import netCDF4
a = netCDF4.Dataset({path!r})            # handle A, stays open
b = netCDF4.Dataset({path!r})            # handle B
b['PRODUCT'].variables['time_utc'][...]  # B reads the vlen string variable
b.close()
c = netCDF4.Dataset({path!r})            # SIGSEGV here
print('no crash')
'''
proc = subprocess.run([sys.executable, '-X', 'faulthandler', '-c', child])
print('return code:', proc.returncode)  # -11 on affected versions

Output on 1.7.4 and 1.7.5.1.post1: return code: -11 (with
Fatal Python error: Segmentation fault from faulthandler).
Output on 1.7.2: no crash / return code: 0.

The crash needs both the HDF5 fill value b'' on the dataset and the
vlen string _FillValue attribute: with only one of the two, there is
no crash.

Native stack (gdb, netCDF4 1.7.5.1.post1)

#0  H5F_addr_decode ()            libhdf5-34cf00cd.so.320.2.0
#1  H5VL__native_blob_specific ()  libhdf5
#2  H5VL_blob_specific ()          libhdf5
#3  H5T__vlen_disk_isnull ()       libhdf5
#4  H5T__conv_vlen ()              libhdf5
#5  H5T_convert_with_ctx ()        libhdf5
#6  H5T_convert ()                 libhdf5
#7  H5D_get_create_plist ()        libhdf5
#8  H5VL__native_dataset_get ()    libhdf5
#9  H5VL_dataset_get ()            libhdf5
#10 H5Dget_create_plist ()         libhdf5
#11 nc4_get_var_meta ()            libnetcdf-3622e713.so.22
#12 nc4_hdf5_find_grp_var_att ()   libnetcdf
#13 NC4_HDF5_inq_var_all ()        libnetcdf
#14 nc_inq_var ()                  libnetcdf
#15 nc_inq_varname ()              libnetcdf
#16 __pyx_f_7netCDF4_8_netCDF4__get_vars    _netCDF4.c:19659
#17 netCDF4._netCDF4.Group.__init__         _netCDF4.c:42799
#23 __pyx_f_7netCDF4_8_netCDF4__get_grps    _netCDF4.c:18983
#24 netCDF4._netCDF4.Dataset.__init__       _netCDF4.c:24689

The crash is in the conversion of the vlen fill value, read when
netCDF-C fetches the dataset creation property list at open.

What is not known

• Not bisected between 1.7.2 and 1.7.4: the first affected wheel, and
  whether the change is in HDF5 (1.14.2 -> 1.14.6) or in netCDF-C
  (4.9.x), is unknown. A test against a plain netCDF-C / HDF5 build
  (without the Python wheel) has not been done.
• The internal mechanism is not verified. Our guess, from the stack and
  the sequence: HDF5 shares one underlying file between the open
  handles, and data cached while B read the vlen strings still points to
  B's file object after B is closed; the next open uses that freed
  pointer.
• Not tested on other platforms (macOS, Windows) or with conda builds.

Workaround used

In our application, every xarray dataset built on a given file shares one
netCDF4.Dataset handle that is closed only when the owning object is
closed, so the file is never reopened while another handle on it is open.