Repository navigation
Segmentation fault when opening already open dataset #1444
Description
Environment
OS: Ubuntu 24.04 (Docker 28.5.1)
Python: 3.12.3
netcdf4-python: 1.7.3 (1.6.5 also tested)
Steps to reproduce (docker)
docker run --rm -it ubuntu:24.04 bashIn the docker container:
apt-get update
DEBIAN_FRONTEND=noninteractive apt-get install -y python3 python3-pip python3-venv netcdf-bin
python3 -m venv venv
source ./venv/bin/activate
pip3 install netCDF4==1.7.3
cat <<EOF > example.cdl
netcdf example {
variables:
string example;
}
EOF
ncgen -4 -b example.cdl
cat <<EOF > main.py
from netCDF4 import Dataset
for i in range(50):
file = Dataset("example.nc", "r")
print("Success")
file.close()
EOFRun:
python3 main.pyOutput:
Segmentation fault (core dumped)Description
Although the code in the example is definitely not recommended, I do not believe that the segmentation error should be the result of its execution.
Some observations:
- No segmentation fault when type
float(instead ofstring) is used in theexample.cdl - I could not reproduce in netcdf-c
- Segmentation fault also for netCDF4-python==1.6.5
- Valgrind log:
==5042== Invalid read of size 1
==5042== at 0x5779194: H5F_addr_decode (in /venv/lib/python3.12/site-packages/netcdf4.libs/libhdf5-49234e76.so.310.5.1)
==5042== by 0x59A4E16: H5VL__native_blob_specific (in /venv/lib/python3.12/site-packages/netcdf4.libs/libhdf5-49234e76.so.310.5.1)
==5042== by 0x599D95E: H5VL_blob_specific (in /venv/lib/python3.12/site-packages/netcdf4.libs/libhdf5-49234e76.so.310.5.1)
==5042== by 0x598C59B: H5T__vlen_disk_isnull (in /venv/lib/python3.12/site-packages/netcdf4.libs/libhdf5-49234e76.so.310.5.1)
==5042== by 0x597D009: H5T__conv_vlen (in /venv/lib/python3.12/site-packages/netcdf4.libs/libhdf5-49234e76.so.310.5.1)
==5042== by 0x58D3490: H5T_convert_with_ctx (in /venv/lib/python3.12/site-packages/netcdf4.libs/libhdf5-49234e76.so.310.5.1)
==5042== by 0x58D3669: H5T_convert (in /venv/lib/python3.12/site-packages/netcdf4.libs/libhdf5-49234e76.so.310.5.1)
==5042== by 0x5740732: H5D_get_create_plist (in /venv/lib/python3.12/site-packages/netcdf4.libs/libhdf5-49234e76.so.310.5.1)
==5042== by 0x59A5EF4: H5VL__native_dataset_get (in /venv/lib/python3.12/site-packages/netcdf4.libs/libhdf5-49234e76.so.310.5.1)
==5042== by 0x59934A3: H5VL_dataset_get (in /venv/lib/python3.12/site-packages/netcdf4.libs/libhdf5-49234e76.so.310.5.1)
==5042== by 0x5714242: H5Dget_create_plist (in /venv/lib/python3.12/site-packages/netcdf4.libs/libhdf5-49234e76.so.310.5.1)
==5042== by 0x5429E4C: nc4_get_var_meta (in /venv/lib/python3.12/site-packages/netcdf4.libs/libnetcdf-306d3f1b.so.22)
==5042== Address 0x40 is not stack'd, malloc'd or (recently) free'd
==5042==
==5042==
==5042== Process terminating with default action of signal 11 (SIGSEGV): dumping core
==5042== Access not within mapped region at address 0x40
==5042== at 0x5779194: H5F_addr_decode (in /venv/lib/python3.12/site-packages/netcdf4.libs/libhdf5-49234e76.so.310.5.1)
==5042== by 0x59A4E16: H5VL__native_blob_specific (in /venv/lib/python3.12/site-packages/netcdf4.libs/libhdf5-49234e76.so.310.5.1)
==5042== by 0x599D95E: H5VL_blob_specific (in /venv/lib/python3.12/site-packages/netcdf4.libs/libhdf5-49234e76.so.310.5.1)
==5042== by 0x598C59B: H5T__vlen_disk_isnull (in /venv/lib/python3.12/site-packages/netcdf4.libs/libhdf5-49234e76.so.310.5.1)
==5042== by 0x597D009: H5T__conv_vlen (in /venv/lib/python3.12/site-packages/netcdf4.libs/libhdf5-49234e76.so.310.5.1)
==5042== by 0x58D3490: H5T_convert_with_ctx (in /venv/lib/python3.12/site-packages/netcdf4.libs/libhdf5-49234e76.so.310.5.1)
==5042== by 0x58D3669: H5T_convert (in /venv/lib/python3.12/site-packages/netcdf4.libs/libhdf5-49234e76.so.310.5.1)
==5042== by 0x5740732: H5D_get_create_plist (in /venv/lib/python3.12/site-packages/netcdf4.libs/libhdf5-49234e76.so.310.5.1)
==5042== by 0x59A5EF4: H5VL__native_dataset_get (in /venv/lib/python3.12/site-packages/netcdf4.libs/libhdf5-49234e76.so.310.5.1)
==5042== by 0x59934A3: H5VL_dataset_get (in /venv/lib/python3.12/site-packages/netcdf4.libs/libhdf5-49234e76.so.310.5.1)
==5042== by 0x5714242: H5Dget_create_plist (in /venv/lib/python3.12/site-packages/netcdf4.libs/libhdf5-49234e76.so.310.5.1)
==5042== by 0x5429E4C: nc4_get_var_meta (in /venv/lib/python3.12/site-packages/netcdf4.libs/libnetcdf-306d3f1b.so.22)
==5042== If you believe this happened as a result of a stack
==5042== overflow in your program's main thread (unlikely but
==5042== possible), you can try to increase the size of the
==5042== main thread stack using the --main-stacksize= flag.
==5042== The main thread stack size used in this run was 8388608.
==5042==
==5042== HEAP SUMMARY:
==5042== in use at exit: 8,131,126 bytes in 11,506 blocks
==5042== total heap usage: 19,922 allocs, 8,416 frees, 48,204,805 bytes allocated
Sorry if duplicated.
Activity
jswhit2 commentedon Nov 4, 2025
Cannot reproduce on macos with
netcdf4-python version: 1.7.3
HDF5 lib version: 1.14.6
netcdf lib version: 4.9.3
numpy version 2.2.5
what versions of netcdf-c and hdf5 do you have?
g2knizn commentedon Nov 5, 2025
Thanks for a quick response.
After some 'debugging' with print statements, I can confirm that the issue is non-deterministic and depends on the behaviour of the garbage collector.
Here is an example of execution that causes segmentation fault (loop 5 times -
for i in range(5)). Each line represents a print statement at the beginning of a function.So the problem occurs when init is called after dealloc is called. When init is called one after another, the problem does not occur. You can increase the probability of segmentation fault by increasing loop range (e.g. 1000)
Segmentation fault (in my case) is in this line:
netcdf4-python/src/netCDF4/_netCDF4.pyx
Line 2536 in a9d4e92
To answer your question:
jswhit commentedon Nov 5, 2025
I can confirm that if I increase the loop iteration count (from 50 to 5000) I do see a segfault.
jswhit commentedon Nov 5, 2025
Looks like the segfault occurs when
nc_inq_varnameis called inside_get_vars. No idea why.