Skip to content

Add mypy as pre-commit hook #26

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Sep 27, 2021
Merged

Conversation

underchemist
Copy link
Contributor

@underchemist underchemist commented Sep 21, 2021

Description

Please include a summary of the change. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes #5

  • Adds mypy as a pre-commit hook
  • Adds mypy step in scripts/cibuild
  • Configuration added to pyproject.toml which requires a more recent version of mypy (v0.812 -> v0.910).
  • Removed py38 options from [tool.black] to support pre-commit in py39 venv
  • Current config produces mypy errors, using ignore_errors config until they can be addressed

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Instructions

Explain what someone needs to do in order to test the functionality of the changes.

Run pre-commit run --all-files or mypy -p oaff

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

  • ran pre-commit locally

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules

@ghost
Copy link

ghost commented Sep 21, 2021

CLA assistant check
All CLA requirements met.

update cibuild script

ignore errors for now until they can be addressed in code/config options

Add note about pre-commit hooks in readme

remove python3.8 language_version
@underchemist
Copy link
Contributor Author

underchemist commented Sep 21, 2021

mypy -p oaff without errors supressed:

mypy output
oaff) ysebastien@oscar:~/git/ogc-api-fast-features$ mypy -p oaff
oaff/app/oaff/app/request_handlers/landing_page.py:15: error: Name "LandingPage" already defined (possibly by an import)
oaff/app/oaff/app/request_handlers/landing_page.py:20: error: Missing return statement
oaff/app/oaff/app/request_handlers/conformance.py:10: error: Name "Conformance" already defined (possibly by an import)
oaff/app/oaff/app/request_handlers/conformance.py:15: error: Missing return statement
oaff/app/oaff/app/data/sources/common/data_source_manager.py:9: error: Method must have at least one argument
oaff/app/oaff/app/responses/response_format.py:6: error: Definition of "__hash__" in base class "dict" is incompatible with definition in base class "Enum"
oaff/app/oaff/app/request_handlers/feature.py:22: error: Missing return statement
oaff/app/oaff/app/request_handlers/collections_list.py:15: error: Name "CollectionsList" already defined (possibly by an import)
oaff/app/oaff/app/request_handlers/collections_list.py:20: error: Missing return statement
oaff/app/oaff/app/request_handlers/collection_items.py:36: error: Missing return statement
oaff/app/oaff/app/request_handlers/collection_items.py:102: error: Incompatible return value type (got "Dict[Any, Any]", expected "List[Any]")
oaff/app/oaff/app/request_handlers/collection_items.py:104: error: Incompatible return value type (got "Callable[[int, int], List[Any]]", expected "Callable[[int, int], Dict[Any, Any]]")
oaff/app/oaff/app/request_handlers/collection_items.py:112: error: Incompatible return value type (got "None", expected "Type[Any]")
oaff/app/oaff/app/request_handlers/collection_items.py:134: error: Too many values to unpack (4 expected, 6 provided)
oaff/app/oaff/app/request_handlers/collection_items.py:134: error: "None" object is not iterable
oaff/app/oaff/app/request_handlers/collection_items.py:135: error: Argument 1 to "len" has incompatible type "Union[Tuple[float, float, float, float], Tuple[float, float, float, float, float, float], None]"; expected "Sized"
oaff/app/oaff/app/request_handlers/collection_items.py:136: error: Value of type "Union[Tuple[float, float, float, float], Tuple[float, float, float, float, float, float], None]" is not indexable
oaff/app/oaff/app/request_handlers/collection_items.py:160: error: Incompatible return value type (got "None", expected "Type[Any]")
oaff/app/oaff/app/request_handlers/collection_items.py:164: error: Need more than 1 value to unpack (2 expected)
oaff/app/oaff/app/request_handlers/collection_items.py:312: error: Incompatible return value type (got "None", expected "Type[Any]")
oaff/app/oaff/app/request_handlers/collection_items.py:338: error: Argument 1 to "astimezone" of "datetime" has incompatible type "Type[tzinfo]"; expected "Optional[tzinfo]"
oaff/app/oaff/app/request_handlers/collection.py:19: error: Missing return statement
oaff/app/oaff/app/responses/templates/templates.py:16: error: "Environment" has no attribute "install_gettext_translations"
oaff/app/oaff/app/data/sources/postgresql/stac_hybrid/postgresql_data_source.py:74: error: "Coroutine[None, None, Database]" not callable
oaff/app/oaff/app/data/sources/postgresql/stac_hybrid/postgresql_data_source.py:183: error: Item "None" of "Optional[Any]" has no attribute "limit"
oaff/app/oaff/app/data/sources/postgresql/stac_hybrid/postgresql_data_source.py:184: error: Item "None" of "Optional[Any]" has no attribute "offset"
oaff/app/oaff/app/data/sources/postgresql/stac_hybrid/postgresql_data_source.py:188: error: Value of type "Optional[Mapping[Any, Any]]" is not indexable
oaff/app/oaff/app/data/sources/postgresql/stac_hybrid/postgresql_data_source.py:226: error: List item 0 has incompatible type "List[None]"; expected "datetime"
oaff/app/oaff/app/data/sources/postgresql/stac_hybrid/postgresql_data_source.py:312: error: Missing return statement
oaff/app/oaff/app/data/sources/postgresql/stac_hybrid/postgresql_data_source.py:418: error: Need type annotation for "table_temporal_extents" (hint: "table_temporal_extents: Dict[<type>, <type>] = ...")
oaff/app/oaff/app/data/sources/postgresql/stac_hybrid/postgresql_data_source.py:441: error: Value of type "Optional[Mapping[Any, Any]]" is not indexable
oaff/fastapi/api/util.py:60: error: Dict entry 0 has incompatible type "str": "int"; expected "str": "str"
oaff/fastapi/api/util.py:61: error: Dict entry 1 has incompatible type "str": "int"; expected "str": "str"
oaff/fastapi/api/routes/common/common_parameters.py:87: error: Incompatible return value type (got "List[str]", expected "Set[str]")
oaff/fastapi/api/routes/common/common_parameters.py:87: error: Argument "key" to "sorted" has incompatible type overloaded function; expected "Callable[[str], SupportsLessThan]"
oaff/fastapi/api/openapi/openapi.py:41: error: Incompatible return value type (got "Callable[[Request], Any]", expected "VndResponse")
oaff/fastapi/api/routes/common/parameter_control.py:12: error: Unsupported left operand type for + ("None")
oaff/fastapi/api/routes/common/parameter_control.py:12: note: Left operand is of type "Optional[List[str]]"
oaff/fastapi/api/routes/collections.py:83: error: Incompatible default for argument "limit_param" (default has type "Query", argument has type "int")
oaff/fastapi/api/routes/collections.py:89: error: Incompatible default for argument "offset_param" (default has type "Query", argument has type "int")
oaff/fastapi/api/routes/collections.py:94: error: Incompatible default for argument "bbox_param" (default has type "Query", argument has type "Optional[str]")
oaff/fastapi/api/routes/collections.py:110: error: Incompatible default for argument "bbox_crs_param" (default has type "Query", argument has type "Optional[str]")
oaff/fastapi/api/routes/collections.py:114: error: Incompatible default for argument "datetime_param" (default has type "Query", argument has type "Optional[str]")
oaff/fastapi/api/routes/collections.py:126: error: Incompatible default for argument "filter_param" (default has type "Query", argument has type "Optional[str]")
oaff/fastapi/api/routes/collections.py:130: error: Incompatible default for argument "filter_lang_param" (default has type "Query", argument has type "Optional[str]")
oaff/fastapi/api/routes/collections.py:134: error: Incompatible default for argument "filter_crs_param" (default has type "Query", argument has type "Optional[str]")
oaff/fastapi/api/routes/collections.py:205: error: Incompatible return value type (got "None", expected "datetime")
oaff/fastapi/api/routes/collections.py:219: error: Incompatible return value type (got "Tuple[datetime, ...]", expected "Union[Tuple[datetime], Tuple[datetime, datetime], None]")
Found 47 errors in 15 files (checked 149 source files)

Comment on lines +15 to +19
rev: v0.910
hooks:
- id: mypy
language_version: python
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain why you decided to move away from the custom hook?
If we keep this approach, ./mypy.sh will not be required and should be deleted.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've bumped mypy==v0.910 since it supports configuration in pyproject.toml which I think removes the need for a custom hook

README.md Outdated
@@ -114,6 +114,9 @@ The following provides information for developers looking to maintain or extend

## Architecture

## Pre-commit hooks
Git hooks are used to run formatting (isort, black), linting (flake8), and type-checking (mypy). You can install the pre-commit scripts with `pre-commit install`. Thereafter, the scripts defined in `.pre-commit-config.yaml` will be run on the files included in a commit. Specific tool configurations are included in `pyproject.toml`.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is useful. Worth mentioning that pre-commit install is run by scripts/setup too

@@ -7,15 +7,13 @@ repos:
rev: 21.5b1
hooks:
- id: black
language_version: python3.8
language_version: python
Copy link
Collaborator

@JBurkinshaw JBurkinshaw Sep 22, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain this and the related change in pyproject.toml a bit more?
Would there be a way to specify a minimum version? i.e. 3.8+

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

specifying python 3.8 here excludes python 3.9 (resulting in error if that is the runtime python version), I'm not sure that we need to enforce a python version here, leaving it to setup.py or README

@underchemist underchemist mentioned this pull request Sep 23, 2021
@underchemist underchemist mentioned this pull request Sep 23, 2021
12 tasks
@JBurkinshaw JBurkinshaw merged commit 3c072f3 into microsoft:main Sep 27, 2021
@underchemist
Copy link
Contributor Author

thanks @JBurkinshaw!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement mypy type validation
2 participants