Skip to content

add sqlalchemy backend #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 22 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
04d22a5
remove other modules
geospatial-jeff Aug 10, 2022
adf7471
remove pgstac from docker/make, local dev working
geospatial-jeff Aug 10, 2022
2b34e16
move python packaging, alembic, and tests to top level of repo
geospatial-jeff Aug 10, 2022
51b6046
squash the package
geospatial-jeff Aug 10, 2022
2993d4f
update paths in dockerfile to new repo structure
geospatial-jeff Aug 10, 2022
0662041
fix paths when running tests locally
geospatial-jeff Aug 10, 2022
131690d
move joplin data to tests/ folder
geospatial-jeff Aug 10, 2022
34e7e22
update github ci
geospatial-jeff Aug 10, 2022
d4cec72
change ci target branch to main
geospatial-jeff Aug 10, 2022
88045a6
run pre-commit
geospatial-jeff Aug 10, 2022
2627d18
add pystac validation to dev dependencies
geospatial-jeff Aug 10, 2022
472540e
fix building of docs, simplify a little
geospatial-jeff Aug 10, 2022
1fb943f
update dependabot config
geospatial-jeff Aug 10, 2022
db261ad
fix deploy docs workflow maybe? not tested yet
geospatial-jeff Aug 10, 2022
ae45b7b
remove publish workflow, this project won't ship a python library
geospatial-jeff Aug 10, 2022
896adc6
Update mkdocs.yml
geospatial-jeff Aug 13, 2022
e86ea6a
postfix container names, network with -sqlalchemy
geospatial-jeff Aug 13, 2022
5f63e88
install pyscopg2 instead of psycopg2-binary
geospatial-jeff Aug 13, 2022
d643679
Merge branch 'add-backend' of https://github.com/stac-utils/stac-fast…
geospatial-jeff Aug 13, 2022
08c3764
add psycopg2 system dependencies to ci and docs container, revert mod…
geospatial-jeff Aug 13, 2022
879cb7d
update readme
geospatial-jeff Aug 13, 2022
58cf32d
rename cicd pipeline
geospatial-jeff Aug 13, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 0 additions & 20 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,26 +8,6 @@ updates:
directory: "/.github/workflows"
schedule:
interval: weekly
- package-ecosystem: pip
directory: "/stac_fastapi/api"
schedule:
interval: weekly
- package-ecosystem: pip
directory: "/stac_fastapi/api"
schedule:
interval: weekly
- package-ecosystem: pip
directory: "/stac_fastapi/types"
schedule:
interval: weekly
- package-ecosystem: pip
directory: "/stac_fastapi/extensions"
schedule:
interval: weekly
- package-ecosystem: pip
directory: "/stac_fastapi/pgstac"
schedule:
interval: weekly
- package-ecosystem: pip
directory: "/stac_fastapi/sqlalchemy"
schedule:
Expand Down
57 changes: 7 additions & 50 deletions .github/workflows/cicd.yaml
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
name: stac-fastapi
name: stac-fastapi-sqlalchemy
on:
push:
branches: [ master ]
branches: [ main ]
pull_request:
branches: [ master ]
branches: [ main ]

jobs:
test:
Expand Down Expand Up @@ -46,33 +46,13 @@ jobs:
- name: Lint code
uses: pre-commit/[email protected]

- name: Install pipenv
- name: Install sqlalchemy backend
run: |
python -m pip install --upgrade pipenv wheel

- name: Install types
run: |
pip install ./stac_fastapi/types[dev]

- name: Install core api
run: |
pip install ./stac_fastapi/api[dev]

- name: Install Extensions
run: |
pip install ./stac_fastapi/extensions[dev]

- name: Install sqlalchemy stac-fastapi
run: |
pip install ./stac_fastapi/sqlalchemy[dev,server]

- name: Install pgstac stac-fastapi
run: |
pip install ./stac_fastapi/pgstac[dev,server]
pip install .[dev,server]

- name: Run migration
run: |
cd stac_fastapi/sqlalchemy && alembic upgrade head
alembic upgrade head
env:
POSTGRES_USER: username
POSTGRES_PASS: password
Expand All @@ -82,19 +62,7 @@ jobs:

- name: Run test suite
run: |
cd stac_fastapi/api && pipenv run pytest -svvv
env:
ENVIRONMENT: testing

- name: Run test suite
run: |
cd stac_fastapi/types && pipenv run pytest -svvv
env:
ENVIRONMENT: testing

- name: Run test suite
run: |
cd stac_fastapi/sqlalchemy && pipenv run pytest -svvv
pytest -svvv
env:
ENVIRONMENT: testing
POSTGRES_USER: username
Expand All @@ -104,17 +72,6 @@ jobs:
POSTGRES_HOST_WRITER: localhost
POSTGRES_PORT: 5432

- name: Run test suite
run: |
cd stac_fastapi/pgstac && pipenv run pytest -svvv
env:
ENVIRONMENT: testing
POSTGRES_USER: username
POSTGRES_PASS: password
POSTGRES_DBNAME: postgis
POSTGRES_HOST_READER: localhost
POSTGRES_HOST_WRITER: localhost
POSTGRES_PORT: 5432
test-docs:
runs-on: ubuntu-latest
steps:
Expand Down
11 changes: 4 additions & 7 deletions .github/workflows/deploy_mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: Publish docs via GitHub Pages
on:
push:
branches:
- master
- main
paths:
# Rebuild website when docs have changed or code has changed
- 'README.md'
Expand All @@ -17,7 +17,7 @@ jobs:
runs-on: ubuntu-latest

steps:
- name: Checkout master
- name: Checkout main
uses: actions/checkout@v3

- name: Set up Python 3.8
Expand All @@ -27,12 +27,9 @@ jobs:

- name: Install dependencies
run: |
sudo apt-get install -y postgresql libpq-dev
python -m pip install --upgrade pip
python -m pip install -e \
stac_fastapi/api \
stac_fastapi/types \
stac_fastapi/extensions \
stac_fastapi/sqlalchemy
python -m pip install -e .
python -m pip install mkdocs mkdocs-material pdocs

- name: update API docs
Expand Down
30 changes: 0 additions & 30 deletions .github/workflows/publish.yml

This file was deleted.

8 changes: 2 additions & 6 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ FROM python:3.8-slim as base
# need the following packages in order to build
RUN apt-get update && \
apt-get -y upgrade && \
apt-get install -y build-essential git && \
apt-get install -y build-essential git postgresql libpq-dev && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*

Expand All @@ -16,8 +16,4 @@ WORKDIR /app

COPY . /app

RUN pip install -e ./stac_fastapi/types[dev] && \
pip install -e ./stac_fastapi/api[dev] && \
pip install -e ./stac_fastapi/extensions[dev] && \
pip install -e ./stac_fastapi/sqlalchemy[dev,server] && \
pip install -e ./stac_fastapi/pgstac[dev,server]
RUN pip install -e .[dev,server]
10 changes: 3 additions & 7 deletions Dockerfile.docs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
FROM python:3.8-slim

# build-essential is required to build a wheel for ciso8601
RUN apt update && apt install -y build-essential
RUN apt update && apt install -y build-essential postgresql libpq-dev

RUN python -m pip install --upgrade pip
RUN python -m pip install mkdocs mkdocs-material pdocs
Expand All @@ -10,16 +10,12 @@ COPY . /opt/src

WORKDIR /opt/src

RUN python -m pip install \
stac_fastapi/api \
stac_fastapi/types \
stac_fastapi/extensions \
stac_fastapi/sqlalchemy
RUN python -m pip install .

CMD ["pdocs", \
"as_markdown", \
"--output_dir", \
"docs/api/", \
"--exclude_source", \
"--overwrite", \
"stac_fastapi"]
"stac_fastapi.sqlalchemy"]
84 changes: 24 additions & 60 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,11 @@
APP_HOST ?= 0.0.0.0
APP_PORT ?= 8080
EXTERNAL_APP_PORT ?= ${APP_PORT}
run_sqlalchemy = docker-compose run --rm \
run_container = docker-compose run --rm \
-p ${EXTERNAL_APP_PORT}:${APP_PORT} \
-e APP_HOST=${APP_HOST} \
-e APP_PORT=${APP_PORT} \
app-sqlalchemy

run_pgstac = docker-compose run --rm \
-p ${EXTERNAL_APP_PORT}:${APP_PORT} \
-e APP_HOST=${APP_HOST} \
-e APP_PORT=${APP_PORT} \
app-pgstac
stac-fastapi

.PHONY: image
image:
Expand All @@ -22,70 +16,40 @@ image:
docker-run-all:
docker-compose up

.PHONY: docker-run-sqlalchemy
docker-run-sqlalchemy: image
$(run_sqlalchemy)
.PHONY: docker-run
docker-run: image
$(run_container)

.PHONY: docker-run-pgstac
docker-run-pgstac: image
$(run_pgstac)
.PHONY: docker-shell
docker-shell:
$(run_container) /bin/bash

.PHONY: docker-shell-sqlalchemy
docker-shell-sqlalchemy:
$(run_sqlalchemy) /bin/bash

.PHONY: docker-shell-pgstac
docker-shell-pgstac:
$(run_pgstac) /bin/bash

.PHONY: test-sqlalchemy
test-sqlalchemy: run-joplin-sqlalchemy
$(run_sqlalchemy) /bin/bash -c 'export && ./scripts/wait-for-it.sh database:5432 && cd /app/stac_fastapi/sqlalchemy/tests/ && pytest -vvv'

.PHONY: test-pgstac
test-pgstac:
$(run_pgstac) /bin/bash -c 'export && ./scripts/wait-for-it.sh database:5432 && cd /app/stac_fastapi/pgstac/tests/ && pytest -vvv'
.PHONY: test
test: run-joplin
$(run_container) /bin/bash -c 'export && ./scripts/wait-for-it.sh database:5432 && pytest -vvv'

.PHONY: test-api
test-api:
$(run_sqlalchemy) /bin/bash -c 'cd /app/stac_fastapi/api && pytest -svvv'

.PHONY: run-database
run-database:
docker-compose run --rm database

.PHONY: run-joplin-sqlalchemy
run-joplin-sqlalchemy:
docker-compose run --rm loadjoplin-sqlalchemy

.PHONY: run-joplin-pgstac
run-joplin-pgstac:
docker-compose run --rm loadjoplin-pgstac

.PHONY: test
test: test-sqlalchemy test-pgstac

.PHONY: pybase-install
pybase-install:
pip install wheel && \
pip install -e ./stac_fastapi/api[dev] && \
pip install -e ./stac_fastapi/types[dev] && \
pip install -e ./stac_fastapi/extensions[dev]

.PHONY: pgstac-install
pgstac-install: pybase-install
pip install -e ./stac_fastapi/pgstac[dev,server]

.PHONY: sqlalchemy-install
sqlalchemy-install: pybase-install
pip install -e ./stac_fastapi/sqlalchemy[dev,server]
.PHONY: run-joplin
run-joplin:
docker-compose run --rm load-joplin

.PHONY: docs-image
docs-image:
docker-compose -f docker-compose.docs.yml \
build
docker build . -f Dockerfile.docs -t stac-fastapi-sqlalchemy-docs:latest

.PHONY: docs
docs: docs-image
docker-compose -f docker-compose.docs.yml \
run docs
docker run --rm \
-v ${CURDIR}/:/opt/src \
-e POSTGRES_USER=username \
-e POSTGRES_PASS=password \
-e POSTGRES_DBNAME=postgis \
-e POSTGRES_HOST_READER=database \
-e POSTGRES_HOST_WRITER=database \
-e POSTGRES_PORT=5432 \
stac-fastapi-sqlalchemy-docs:latest
Loading