Repository navigation
Job with many results: read timeout on metadata #462
Description
So one LCFM job has this read timeout on result metadata, the job generated many results:
url = '/openeo/1.2/jobs/j-26013001053549dc9fbe27b7f0c25299/results
https://jenkins.vgt.vito.be/job/openEO/job/openeo-apex-benchmarks/418/testReport/junit/tests/test_b…
https://openeo-admin.vgt.vito.be/job/j-26013001053549dc9fbe27b7f0c25299
Not a blocker, but requires further investigation:
Does request time increase with size of the result? Do we need different timeout?
What about stac 1.1 codepath, is that better?
Anything we can fix? (But avoiding deep dive into 1.0 codepaths...)
Metadata
Metadata
Assignees
Labels
No labels
Activity
jdries commentedon Jan 30, 2026
@JeroenVerstraelen assigned for triage and prio planning.
It causes false positives in apex tests, and users can also experience this.
JeroenVerstraelen commentedon Feb 11, 2026
Context
s3://openeo-data-staging-waw4-1/batch_jobs/j-26013001053549dc9fbe27b7f0c25299/job_metadata.jsonContains 312 items in the
.assets[]property.This causes a timeout in the following request:
GET https://openeo-staging.dataspace.copernicus.eu/openeo/1.2/jobs/j-26013001053549dc9fbe27b7f0c25299/results
Does request time increase with size of the result? Do we need different timeout?
- Yes, due to file I/O stats and URL signing per asset (312 assets × ~100ms ≈ 31s); timeout likely needs increasing.
What about stac 1.1 codepath, is that better?
-
Yes, when result_metadata.items is populated. It returns a lightweight Collection with item links instead of embedding all 312 assets inline.Edit: stac 1.1 will codepath will provide no improvement.Anything we can fix? (But avoiding deep dive into 1.0 codepaths...)
- pre-compute file:size during job completion (not at request time) to avoid file I/O at request time, increase timeout for large result sets.
JeroenVerstraelen commentedon Feb 11, 2026
Added to planning backlog.
bossie commentedon Feb 11, 2026
The STAC 1.1
Collectiondocument also embeds these assets I'm afraid, so this makes no difference. This is how the API mandates it and thus how the clients download these assets: they find the assets in a top-levelassetsproperty* rather than walk the STAC tree.There could potentially be less
itemlinks in the STAC 1.1Collectiondocument because there is no longer a 1:1 mapping of asset to item but I think these are edge cases really (think:separate_assets_per_band).*this was actually a very real issue that messed up the download of STAC 1.1 assets. Because the top-level
assetsproperty is a dictionary with unique keys, and the STAC 1.1 implementation has unified asset keys across items, you can't have multiple assets with the same asset key in this dictionary. The solution there was to use{item_id}_{asset_key}as the dictionary key instead.EmileSonneveld commentedon Mar 31, 2026
file:size can be logged as a scala issue
jdries commentedon Oct 12, 2026
@EmileSonneveld @JeroenVerstraelen this issue mentions a simple change in timeout value as potential quick solution to a real issue, did that happen already? (No evidence provided in this issue.)