Skip to content

Add parcel delineation UDP #197

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 12 commits into
base: main
Choose a base branch
from
Draft

Add parcel delineation UDP #197

wants to merge 12 commits into from

Conversation

manugv
Copy link
Collaborator

@manugv manugv commented Jun 3, 2025

Add parcel delineation UDP.

The UDP is generated from the openeo community example.

@manugv manugv marked this pull request as draft June 3, 2025 09:50
@manugv manugv requested a review from jdries June 3, 2025 09:50
@manugv manugv requested a review from soxofaan June 3, 2025 12:14
@soxofaan
Copy link
Contributor

soxofaan commented Jun 4, 2025

unit tests still fail:
e.g. see https://github.com/ESA-APEx/apex_algorithms/actions/runs/15419058077/job/43388958275

__________________ test_record_validation[parcel_delineation] __________________
...
>           raise error
E           jsonschema.exceptions.ValidationError: ['http://www.opengis.net/spec/ogcapi-records-1/1.0/req/record-core'] is too short
E           
E           Failed validating 'minItems' in schema['properties']['conformsTo']:
E               {'type': 'array',
E                'description': 'The extensions/conformance classes used in this '
E                               'record.',
E                'items': {'type': 'string'},
E                'minItems': 2,
E                'uniqueItems': True,
E                'anyOf': [{'contains': {'const': 'http://www.opengis.net/spec/ogcapi-records-1/1.0/req/record-core'}}],
E                'oneOf': [{'contains': {'const': 'https://apex.esa.int/core/openeo-udp'}},
E                          {'contains': {'const': 'https://apex.esa.int/core/ogc-api-processes'}}]}
E           
E           On instance['conformsTo']:
E               ['http://www.opengis.net/spec/ogcapi-records-1/1.0/req/record-core']

you should also list https://apex.esa.int/core/openeo-udp under conformsTo of algorithm_catalog/vito/parcel_delineation/records/parcel_delineation.json

"process_graph": {
"parcel_delineation1": {
"process_id": "parcel_delineation",
"namespace": "https://github.com/raw/ESA-APEx/apex_algorithms/refs/heads/main/algorithm_catalog/vito/parcel_delineation/openeo_udp/parcel_delineation.json",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this URL does not exist (yet) which causes this unit test to fail:

____________________ test_lint_scenario[parcel_delineation] ____________________

scenario = BenchmarkScenario(id='parcel_delineation', description='Parcel Delineation based on ML using Sentinal-2', backend='ope...al_extent': ['2021-01-01', '2021-12-31']}, 'result': True}}, job_options=None, reference_data={}, reference_options={})

    @pytest.mark.parametrize(
        "scenario",
        [
            # Use scenario id as parameterization id to give nicer test names.
            pytest.param(uc, id=uc.id)
            for uc in get_benchmark_scenarios()
        ],
    )
    def test_lint_scenario(scenario: BenchmarkScenario):
>       lint_benchmark_scenario(scenario)

tests/test_scenarios.py:27: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/opt/hostedtoolcache/Python/3.12.10/x64/lib/python3.12/site-packages/apex_algorithm_qa_tools/scenarios.py:109: in lint_benchmark_scenario
    resp.raise_for_status()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Response [404]>

    def raise_for_status(self):
        """Raises :class:`HTTPError`, if one occurred."""
    
        http_error_msg = ""
        if isinstance(self.reason, bytes):
            # We attempt to decode utf-8 first because some servers
            # choose to localize their reason strings. If the string
            # isn't utf-8, we fall back to iso-8859-1 for all other
            # encodings. (See PR #3538)
            try:
                reason = self.reason.decode("utf-8")
            except UnicodeDecodeError:
                reason = self.reason.decode("iso-8859-1")
        else:
            reason = self.reason
    
        if 400 <= self.status_code < 500:
            http_error_msg = (
                f"{self.status_code} Client Error: {reason} for url: {self.url}"
            )
    
        elif 500 <= self.status_code < 600:
            http_error_msg = (
                f"{self.status_code} Server Error: {reason} for url: {self.url}"
            )
    
        if http_error_msg:
>           raise HTTPError(http_error_msg, response=self)
E           requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/raw/ESA-APEx/apex_algorithms/refs/heads/main/algorithm_catalog/vito/parcel_delineation/openeo_udp/parcel_delineation.json

you can make it pass by updating this to the pinned commit in the feature branch:
https://github.com/raw/ESA-APEx/apex_algorithms/e5d5e1128b091f3086adb7ed31e6d9e755087097/algorithm_catalog/vito/parcel_delineation/records/parcel_delineation.json

Copy link
Collaborator Author

@manugv manugv Jun 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yesterday I tried it and it failed. I believe the test only allows main/master branch.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it only allows main/master if you use a branch based URL (something with .../refs/heads/....)

But if you use a direct commit reference, that check is skipped.

For example, directly from the github PR UI:

this is an URL that should be accepted

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note that you can also construct such an URL by hand by just filling in the parts appropriately:
https://github.com/raw/$ORG/$REPO/$COMMITHASH/$FILEPATH

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this procedure or these tricks should be documented somewhere. Or they already are?

anybody some insights about that, @HansVRP @Pratichhya ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I don't think we have documented these tricks anywhere.

Documenting these, as well as the standards/tricks when handling UDF/dependencies for JSONs in this repo, can be done in a similar location as the knowledge sharing location (yet to finalise) isn't it @HansVRP.

Copy link
Contributor

@HansVRP HansVRP Jun 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add it to the apex documentation? developers who want to onboard their service, should be able to find a straightforward guide on how to do so.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants