Phase out read_vector usage

@jdries

when passing a string as polygon argument to ImageCollectionClient.mask:

openeo-python-client/openeo/rest/imagecollectionclient.py

Lines 777 to 784 in b5f3b47

    
           if isinstance(polygon, str): 
        
               new_collection = self.graph_add_process('read_vector', args={ 
        
                   'filename': polygon 
        
               }) 
        
               mask = { 
        
                   'from_node': new_collection.node_id 
        
               }

or in _get_geometry_argument:

openeo-python-client/openeo/rest/datacube.py

Line 900 in 2ce9d3b

# TODO #104: `read_vector` is non-standard process.

this implementation assumes process read_vector which is currently a VITO specific process and I'm not aware of anything alike in the official process collection.

Background: At VITO we currently use this to work with very large polygon files (> 100k polygons) stored at backend side, which we don't want to pass with the openEO request for obvious reasons.

In the client we should avoid hardcoding non-official processes of course

Solution:

check which processes are supported by backend: read_vector, load_url, load_geojson
check if the string is an http url, a path to geojson that exists locally, or a path that does not exist locally
depending on the combination of available processes and the type of string, do something that's more sensible than current implementation
Some care should be taken to avoid passing a huge geojson object with the request (e.g. throw an exception when above a threshold)
Still allow the possibility to inject a read_vector based argument to support the VITO use cases

cc @jdries

A fix for this is being proposed in this pull request:
Open-EO/openeo-processes#106

ah nice, I somehow missed that thread

however, load_uploaded_files is meant for user uploaded files, which is not exactly what we currently do in the VITO use cases that depend on read_vector.

A closer proposal is probably the import_nfs process from Open-EO/openeo-processes#105

way forward:

Support load_geojson and load_url #457

@EmileSonneveld

Another use case that has to be updated: this suggestion was made in a user support channel to do aggregate_spatial with a (large) geometry from a URL:

datacube = datacube.aggregate_spatial(
    geometries="https://example.com/path/to/geometries.json",
    reducer="mean",
)

will currently produce a process graph using read_vector, so that's not future proof at the moment

(cc @EmileSonneveld)

Issue #104/#457 replace `read_vector` with `load_url` in `DataCube._g…

Issue #457/#104/#672 add automatic `load_url` support

Better argument handling in _get_geometry_argument

Issue #104 drop usage of non-standard read_vector process

Issue #104 add docs on how to reconstruct `read_vector` for old workf…

Better argument handling in _get_geometry_argument

Issue #104 drop usage of non-standard read_vector process

Issue #104 add docs on how to reconstruct `read_vector` for old workf…

done:

remove read_vector usage from default geometry handling (but documented how to reconstruct for workflows that still need it)
add support for passing a geometry URL

Navigation Menu

Repository navigation

Phase out read_vector usage #104

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	if isinstance(polygon, str):
	new_collection = self.graph_add_process('read_vector', args={
	'filename': polygon
	})

	mask = {
	'from_node': new_collection.node_id
	}

Phase out read_vector usage #104

Description

Activity

jdries commented on Dec 17, 2019

soxofaan commented on Dec 17, 2019

soxofaan commented on Dec 17, 2019

soxofaan commented on Jul 18, 2024

soxofaan commented on Oct 24, 2024

soxofaan commented on Nov 27, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions