ciso8601 as an optional dependency

[keul]

We have currently an issue with stac_fastapi 2.4 in Python 3.10 conda environment due to the recent introduction on ciso8601 as a dependency: see #368

What about having this dependency optional? ciso8601 is a C dependency, so a bit trickier to be installed and it's only an optimisation.

[vincentsarago]

ref #204 (comment)

[geospatial-jeff]

What type of issue are you seeing? Including traceback would be helpful.

[geospatial-jeff]

I'm unable to replicate this on my machine (using miniconda):

conda create -n py310 python=3.10
conda activate py310
pip install stac-fastapi.sqlalchemy

Output:

Collecting stac-fastapi.sqlalchemy
  Downloading stac_fastapi.sqlalchemy-2.4.0-py3-none-any.whl (34 kB)
Collecting stac-fastapi.extensions
  Downloading stac_fastapi.extensions-2.4.0-py3-none-any.whl (16 kB)
Collecting attrs
  Using cached attrs-22.1.0-py2.py3-none-any.whl (58 kB)
Collecting geoalchemy2<0.8.0
  Downloading GeoAlchemy2-0.7.0-py2.py3-none-any.whl (21 kB)
Collecting stac-fastapi.api
  Downloading stac_fastapi.api-2.4.0-py3-none-any.whl (12 kB)
Collecting stac-pydantic>=2.0.3
  Using cached stac_pydantic-2.0.3-py3-none-any.whl (17 kB)
Collecting sqlalchemy==1.3.23
  Downloading SQLAlchemy-1.3.23.tar.gz (6.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.3/6.3 MB 3.8 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting alembic
  Downloading alembic-1.8.1-py3-none-any.whl (209 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 209.8/209.8 kB 3.2 MB/s eta 0:00:00
Collecting shapely
  Using cached Shapely-1.8.2-cp310-cp310-win_amd64.whl (1.3 MB)
Collecting pydantic[dotenv]
  Using cached pydantic-1.9.1-cp310-cp310-win_amd64.whl (2.0 MB)
Collecting psycopg2-binary
  Downloading psycopg2_binary-2.9.3-cp310-cp310-win_amd64.whl (1.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 3.4 MB/s eta 0:00:00
Collecting stac-fastapi.types
  Downloading stac_fastapi.types-2.4.0-py3-none-any.whl (13 kB)
Collecting sqlakeyset
  Downloading sqlakeyset-1.0.1659142803-py3-none-any.whl (20 kB)
Collecting fastapi-utils
  Downloading fastapi_utils-0.2.1-py3-none-any.whl (16 kB)
Collecting click
  Using cached click-8.1.3-py3-none-any.whl (96 kB)
Collecting geojson-pydantic
  Using cached geojson_pydantic-0.4.3-py3-none-any.whl (7.9 kB)
Collecting Mako
  Downloading Mako-1.2.1-py3-none-any.whl (78 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.6/78.6 kB 4.3 MB/s eta 0:00:00
Collecting fastapi
  Downloading fastapi-0.79.0-py3-none-any.whl (54 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.6/54.6 kB ? eta 0:00:00
Collecting typing-extensions>=3.7.4.3
  Using cached typing_extensions-4.3.0-py3-none-any.whl (25 kB)
Collecting python-dotenv>=0.10.4
  Using cached python_dotenv-0.20.0-py3-none-any.whl (17 kB)
Collecting packaging>=20.0
  Using cached packaging-21.3-py3-none-any.whl (40 kB)
Collecting python-dateutil
  Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting brotli-asgi
  Using cached brotli_asgi-1.1.0-py3-none-any.whl (4.3 kB)
Collecting ciso8601~=2.2.0
  Using cached ciso8601-2.2.0.tar.gz (18 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting pystac==1.*
  Downloading pystac-1.6.1-py3-none-any.whl (146 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 146.0/146.0 kB 2.9 MB/s eta 0:00:00
Collecting starlette==0.19.1
  Downloading starlette-0.19.1-py3-none-any.whl (63 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.3/63.3 kB 3.3 MB/s eta 0:00:00
Collecting anyio<5,>=3.4.0
  Using cached anyio-3.6.1-py3-none-any.whl (80 kB)
Collecting pyparsing!=3.0.5,>=2.0.2
  Using cached pyparsing-3.0.9-py3-none-any.whl (98 kB)
Collecting six>=1.5
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting brotli>=1.0.7
  Downloading Brotli-1.0.9-cp310-cp310-win_amd64.whl (383 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 383.3/383.3 kB 3.4 MB/s eta 0:00:00
Collecting colorama
  Using cached colorama-0.4.5-py2.py3-none-any.whl (16 kB)
Collecting MarkupSafe>=0.9.2
  Using cached MarkupSafe-2.1.1-cp310-cp310-win_amd64.whl (17 kB)
Collecting idna>=2.8
  Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting sniffio>=1.1
  Using cached sniffio-1.2.0-py3-none-any.whl (10 kB)
Building wheels for collected packages: sqlalchemy, ciso8601
  Building wheel for sqlalchemy (setup.py) ... done
  Created wheel for sqlalchemy: filename=SQLAlchemy-1.3.23-cp310-cp310-win_amd64.whl size=1221724 sha256=6229034aba102b3725b9bb3cbf4cfd65e2aa74912a54f75661d6ac121cb18232
  Stored in directory: c:\users\jeffa\appdata\local\pip\cache\wheels\6b\bd\5f\b63dc9976fc54a34ac2019fdbef4eb6a15aef4958298ce4b55
  Building wheel for ciso8601 (pyproject.toml) ... done
  Created wheel for ciso8601: filename=ciso8601-2.2.0-cp310-cp310-win_amd64.whl size=14986 sha256=77476234792bda11e7514618bec4ca23824f131f41824fa6771489fcbbff5fba
  Stored in directory: c:\users\jeffa\appdata\local\pip\cache\wheels\02\1f\3b\a3fc12fc79c99e80e030beb98f07c92cbf1e43e00a7abccfcb
Successfully built sqlalchemy ciso8601
Installing collected packages: ciso8601, brotli, typing-extensions, sqlalchemy, sniffio, six, shapely, python-dotenv, pyparsing, psycopg2-binary, MarkupSafe, idna, colorama, attrs, python-dateutil, pydantic, packaging, Mako, geoalchemy2, click, anyio, starlette, sqlakeyset, pystac, geojson-pydantic, alembic, stac-pydantic, fastapi, brotli-asgi, stac-fastapi.types, fastapi-utils, stac-fastapi.api, stac-fastapi.extensions, stac-fastapi.sqlalchemy
Successfully installed Mako-1.2.1 MarkupSafe-2.1.1 alembic-1.8.1 anyio-3.6.1 attrs-22.1.0 brotli-1.0.9 brotli-asgi-1.1.0 ciso8601-2.2.0 click-8.1.3 colorama-0.4.5 fastapi-0.79.0 fastapi-utils-0.2.1 geoalchemy2-0.7.0 geojson-pydantic-0.4.3 idna-3.3 packaging-21.3 psycopg2-binary-2.9.3 pydantic-1.9.1 pyparsing-3.0.9 pystac-1.6.1 python-dateutil-2.8.2 python-dotenv-0.20.0 shapely-1.8.2 six-1.16.0 sniffio-1.2.0 sqlakeyset-1.0.1659142803 sqlalchemy-1.3.23 stac-fastapi.api-2.4.0 stac-fastapi.extensions-2.4.0 stac-fastapi.sqlalchemy-2.4.0 stac-fastapi.types-2.4.0 stac-pydantic-2.0.3 starlette-0.19.1 typing-extensions-4.3.0

[keul]

I missed another information: everything is inside a docker image based on continuumio/miniconda3

@geospatial-jeff

#13 98.81 Installing pip dependencies: ...working... Pip subprocess error:
#13 108.2   error: subprocess-exited-with-error
#13 108.2
#13 108.2   × Building wheel for ciso8601 (pyproject.toml) did not run successfully.
#13 108.2   │ exit code: 1
#13 108.2   ╰─> [13 lines of output]
#13 108.2       running bdist_wheel
#13 108.2       running build
#13 108.2       running build_py
#13 108.2       creating build
#13 108.2       creating build/lib.linux-x86_64-cpython-310
#13 108.2       creating build/lib.linux-x86_64-cpython-310/ciso8601
#13 108.2       copying ciso8601/__init__.pyi -> build/lib.linux-x86_64-cpython-310/ciso8601
#13 108.2       copying ciso8601/py.typed -> build/lib.linux-x86_64-cpython-310/ciso8601
#13 108.2       running build_ext
#13 108.2       building 'ciso8601' extension
#13 108.2       creating build/temp.linux-x86_64-cpython-310
#13 108.2       gcc -pthread -B /opt/conda/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /opt/conda/include -fPIC -O2 -isystem /opt/conda/include -fPIC -DCISO8601_VERSION=2.2.0 -DCISO8601_CACHING_ENABLED=1 -I/opt/conda/include/python3.10 -c module.c -o build/temp.linux-x86_64-cpython-310/module.o
#13 108.2       error: command 'gcc' failed: No such file or directory
#13 108.2       [end of output]
#13 108.2
#13 108.2   note: This error originates from a subprocess, and is likely not a problem with pip.
#13 108.2   ERROR: Failed building wheel for ciso8601
#13 108.2 ERROR: Could not build wheels for ciso8601, which is required to install pyproject.toml-based projects
#13 108.2

[vincentsarago]

error: command 'gcc' failed: No such file or directory

install gcc on your image 🤷‍♂️

[geospatial-jeff]

It looks like that image doesn't have gcc so its unable to build the wheel, can you try installing build-essential?

sudo apt-get install build-essential

https://github.com/stac-utils/stac-fastapi/blob/master/Dockerfile#L5-L9

[duckontheweb]

I still get an error after installing build-essential. This seems like a known issue with building ciso8601 in some environments (see closeio/ciso8601#123(comment).

I'll see if I can figure out a workaround for this.

[geospatial-jeff]

Looks like python 3.10 support is added in ciso8601==2.2.1 which was PRd but never merged closeio/ciso8601#115. This release also adds wheels that @kylebarron made to make it easier to install (closeio/ciso8601#109).

[geospatial-jeff]

So at this point I'd like to wait to see if the new version of ciso8601 is released. We have the option of switching to pyiso8601 but its not a complete RFC3339 parser (while ciso8601 is).

https://github.com/radiantearth/stac-api-spec/blob/main/implementation.md#datetime-parameter-handling

[duckontheweb]

I still get an error after installing build-essential. This seems like a known issue with building ciso8601 in some environments (see closeio/ciso8601#123(comment).

I'll see if I can figure out a workaround for this.

This particular error seems to be specific to Python 3.9 in the continuumio/miniconda3:latest image and it seems like it stems from the fact that that version of Python sets the -n1 compile flag when building C extensions, which gcc doesn't like:

(base) root@635c876df361:/opt/conda# python -c "from sysconfig import get_config_var; print(get_config_var(\"CFLAGS\"));"
-Wno-unused-result -Wsign-compare -DNDEBUG  -O2 -Wall  -fPIC   -O2  -n1 .2-a+fp16+rcpc+dotprod+crypto -isystem /opt/conda/include -I/opt/conda/include  -fPIC   -O2  -n1 .2-a+fp16+rcpc+dotprod+crypto -isystem /opt/conda/include

Creating a Python 3.8 or 3.10 environment allows me to install the library without a problem:

(test) root@635c876df361:/opt/conda# conda create --name test-310 python=3.10
(test) root@635c876df361:/opt/conda# conda activate test-310
(test-310) root@635c876df361:/opt/conda# python -c "from sysconfig import get_config_var; print(get_config_var(\"CFLAGS\"));"
-Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC   -O3  -isystem /opt/conda/envs/test-310/include -fPIC   -O3  -isystem /opt/conda/envs/test-310/include
(test-310) root@635c876df361:/opt/conda# pip install ciso8601
Collecting ciso8601
  Using cached ciso8601-2.2.0.tar.gz (18 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: ciso8601
  Building wheel for ciso8601 (pyproject.toml) ... done
  Created wheel for ciso8601: filename=ciso8601-2.2.0-cp310-cp310-linux_aarch64.whl size=29928 sha256=040a6185a7d8e1857e9edd3bef856225a647ee39f8c604ba75bca2867229837e
  Stored in directory: /root/.cache/pip/wheels/02/1f/3b/a3fc12fc79c99e80e030beb98f07c92cbf1e43e00a7abccfcb
Successfully built ciso8601
Installing collected packages: ciso8601
Successfully installed ciso8601-2.2.0

[keul]

There's also ciso8601 on conda. I'll try that before adding gcc

[geospatial-jeff]

@keul The ciso8601 dependency is removed in 2.4.1