Skip to content

chore: use riot for django tests #1760

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Nov 10, 2020
Merged

chore: use riot for django tests #1760

merged 5 commits into from
Nov 10, 2020

Conversation

majorgreys
Copy link
Contributor

@majorgreys majorgreys commented Oct 30, 2020 •
edited
Loading

Description

Replaces tox with riot for django tests.

Also re-enables migration tests which were mistakenly missed with changes in #1744 with filter on ^django_contrib- rather than ^django_contrib which would capture django_contrib_migration.

Total duration of the django CI job goes down from 26 minutes to 11 minutes (before parallelism) and 10 mins to 5 mins (after parallelism).

List of tox envs (=69):

$ tox -l | grep django
django_contrib-py27-django18-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib-py27-django111-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib-py35-django18-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib-py35-django111-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib-py36-django18-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib-py36-django111-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib_migration-py27-django18-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib_migration-py27-django111-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib_migration-py35-django18-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib_migration-py35-django111-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib_migration-py36-django18-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib_migration-py36-django111-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib-py35-django20-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib-py35-django21-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib-py35-django22-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib_migration-py35-django20-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib_migration-py35-django21-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib_migration-py35-django22-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib-py36-django20-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib-py36-django21-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib-py36-django22-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib-py36-django30-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib-py36-django-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib-py37-django20-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib-py37-django21-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib-py37-django22-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib-py37-django30-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib-py37-django-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib-py38-django20-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib-py38-django21-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib-py38-django22-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib-py38-django30-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib-py38-django-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib_migration-py36-django20-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib_migration-py36-django21-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib_migration-py36-django22-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib_migration-py36-django30-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib_migration-py36-django-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib_migration-py37-django20-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib_migration-py37-django21-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib_migration-py37-django22-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib_migration-py37-django30-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib_migration-py37-django-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib_migration-py38-django20-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib_migration-py38-django21-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib_migration-py38-django22-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib_migration-py38-django30-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_contrib_migration-py38-django-djangopylibmc06-djangoredis45-pylibmc-redis210-memcached-test_django
django_drf_contrib-py27-django111-djangorestframework34-test_django
django_drf_contrib-py27-django111-djangorestframework37-test_django
django_drf_contrib-py35-django111-djangorestframework34-test_django
django_drf_contrib-py35-django111-djangorestframework37-test_django
django_drf_contrib-py36-django111-djangorestframework34-test_django
django_drf_contrib-py36-django111-djangorestframework37-test_django
django_drf_contrib-py35-django22-djangorestframework38-test_django
django_drf_contrib-py35-django22-djangorestframework310-test_django
django_drf_contrib-py35-django22-djangorestframework-test_django
django_drf_contrib-py36-django22-djangorestframework38-test_django
django_drf_contrib-py36-django22-djangorestframework310-test_django
django_drf_contrib-py36-django22-djangorestframework-test_django
django_drf_contrib-py37-django22-djangorestframework38-test_django
django_drf_contrib-py37-django22-djangorestframework310-test_django
django_drf_contrib-py37-django22-djangorestframework-test_django
django_drf_contrib-py36-django30-djangorestframework310-test_django
django_drf_contrib-py37-django30-djangorestframework310-test_django
django_drf_contrib-py38-django30-djangorestframework310-test_django
django_drf_contrib-py36-django-djangorestframework311-test_django
django_drf_contrib-py37-django-djangorestframework311-test_django
django_drf_contrib-py38-django-djangorestframework311-test_django

List of riot cases (=69):

$ riot list | grep "django"
  Python 2.7 'django>=1.8,<1.9' 'django-pylibmc>=0.6,<0.7' 'django-redis>=4.5,<4.6' 'pylibmc' 'pytest-django==3.10.0' 'python-memcached' 'redis>=2.10,<2.11'
  Python 2.7 'django>=1.11,<1.12' 'django-pylibmc>=0.6,<0.7' 'django-redis>=4.5,<4.6' 'pylibmc' 'pytest-django==3.10.0' 'python-memcached' 'redis>=2.10,<2.11'
  Python 3.5 'django>=1.8,<1.9' 'django-pylibmc>=0.6,<0.7' 'django-redis>=4.5,<4.6' 'pylibmc' 'pytest-django==3.10.0' 'python-memcached' 'redis>=2.10,<2.11'
  Python 3.5 'django>=1.11,<1.12' 'django-pylibmc>=0.6,<0.7' 'django-redis>=4.5,<4.6' 'pylibmc' 'pytest-django==3.10.0' 'python-memcached' 'redis>=2.10,<2.11'
  Python 3.6 'django>=1.8,<1.9' 'django-pylibmc>=0.6,<0.7' 'django-redis>=4.5,<4.6' 'pylibmc' 'pytest-django==3.10.0' 'python-memcached' 'redis>=2.10,<2.11'
  Python 3.6 'django>=1.11,<1.12' 'django-pylibmc>=0.6,<0.7' 'django-redis>=4.5,<4.6' 'pylibmc' 'pytest-django==3.10.0' 'python-memcached' 'redis>=2.10,<2.11'
  Python 3.5 'django>=2.0,<2.1' 'django-pylibmc>=0.6,<0.7' 'django-redis>=4.5,<4.6' 'pylibmc' 'pytest-django==3.10.0' 'python-memcached' 'redis>=2.10,<2.11'
  Python 3.5 'django>=2.1,<2.2' 'django-pylibmc>=0.6,<0.7' 'django-redis>=4.5,<4.6' 'pylibmc' 'pytest-django==3.10.0' 'python-memcached' 'redis>=2.10,<2.11'
  Python 3.5 'django>=2.2,<2.3' 'django-pylibmc>=0.6,<0.7' 'django-redis>=4.5,<4.6' 'pylibmc' 'pytest-django==3.10.0' 'python-memcached' 'redis>=2.10,<2.11'
  Python 3.6 'django>=2.0,<2.1' 'django-pylibmc>=0.6,<0.7' 'django-redis>=4.5,<4.6' 'pylibmc' 'pytest-django==3.10.0' 'python-memcached' 'redis>=2.10,<2.11'
  Python 3.6 'django>=2.1,<2.2' 'django-pylibmc>=0.6,<0.7' 'django-redis>=4.5,<4.6' 'pylibmc' 'pytest-django==3.10.0' 'python-memcached' 'redis>=2.10,<2.11'
  Python 3.6 'django>=2.2,<2.3' 'django-pylibmc>=0.6,<0.7' 'django-redis>=4.5,<4.6' 'pylibmc' 'pytest-django==3.10.0' 'python-memcached' 'redis>=2.10,<2.11'
  Python 3.6 'django>=3.0,<3.1' 'django-pylibmc>=0.6,<0.7' 'django-redis>=4.5,<4.6' 'pylibmc' 'pytest-django==3.10.0' 'python-memcached' 'redis>=2.10,<2.11'
  Python 3.6 'django' 'django-pylibmc>=0.6,<0.7' 'django-redis>=4.5,<4.6' 'pylibmc' 'pytest-django==3.10.0' 'python-memcached' 'redis>=2.10,<2.11'
  Python 3.7 'django>=2.0,<2.1' 'django-pylibmc>=0.6,<0.7' 'django-redis>=4.5,<4.6' 'pylibmc' 'pytest-django==3.10.0' 'python-memcached' 'redis>=2.10,<2.11'
  Python 3.7 'django>=2.1,<2.2' 'django-pylibmc>=0.6,<0.7' 'django-redis>=4.5,<4.6' 'pylibmc' 'pytest-django==3.10.0' 'python-memcached' 'redis>=2.10,<2.11'
  Python 3.7 'django>=2.2,<2.3' 'django-pylibmc>=0.6,<0.7' 'django-redis>=4.5,<4.6' 'pylibmc' 'pytest-django==3.10.0' 'python-memcached' 'redis>=2.10,<2.11'
  Python 3.7 'django>=3.0,<3.1' 'django-pylibmc>=0.6,<0.7' 'django-redis>=4.5,<4.6' 'pylibmc' 'pytest-django==3.10.0' 'python-memcached' 'redis>=2.10,<2.11'
  Python 3.7 'django' 'django-pylibmc>=0.6,<0.7' 'django-redis>=4.5,<4.6' 'pylibmc' 'pytest-django==3.10.0' 'python-memcached' 'redis>=2.10,<2.11'
  Python 3.8 'django>=2.0,<2.1' 'django-pylibmc>=0.6,<0.7' 'django-redis>=4.5,<4.6' 'pylibmc' 'pytest-django==3.10.0' 'python-memcached' 'redis>=2.10,<2.11'
  Python 3.8 'django>=2.1,<2.2' 'django-pylibmc>=0.6,<0.7' 'django-redis>=4.5,<4.6' 'pylibmc' 'pytest-django==3.10.0' 'python-memcached' 'redis>=2.10,<2.11'
  Python 3.8 'django>=2.2,<2.3' 'django-pylibmc>=0.6,<0.7' 'django-redis>=4.5,<4.6' 'pylibmc' 'pytest-django==3.10.0' 'python-memcached' 'redis>=2.10,<2.11'
  Python 3.8 'django>=3.0,<3.1' 'django-pylibmc>=0.6,<0.7' 'django-redis>=4.5,<4.6' 'pylibmc' 'pytest-django==3.10.0' 'python-memcached' 'redis>=2.10,<2.11'
  Python 3.8 'django' 'django-pylibmc>=0.6,<0.7' 'django-redis>=4.5,<4.6' 'pylibmc' 'pytest-django==3.10.0' 'python-memcached' 'redis>=2.10,<2.11'
 TEST_DATADOG_DJANGO_MIGRATION=1 Python 2.7 'pytest-django==3.10.0' 'django>=1.8,<1.9'
 TEST_DATADOG_DJANGO_MIGRATION=1 Python 2.7 'pytest-django==3.10.0' 'django>=1.11,<1.12'
 TEST_DATADOG_DJANGO_MIGRATION=1 Python 3.5 'pytest-django==3.10.0' 'django>=1.8,<1.9'
 TEST_DATADOG_DJANGO_MIGRATION=1 Python 3.5 'pytest-django==3.10.0' 'django>=1.11,<1.12'
 TEST_DATADOG_DJANGO_MIGRATION=1 Python 3.6 'pytest-django==3.10.0' 'django>=1.8,<1.9'
 TEST_DATADOG_DJANGO_MIGRATION=1 Python 3.6 'pytest-django==3.10.0' 'django>=1.11,<1.12'
 TEST_DATADOG_DJANGO_MIGRATION=1 Python 3.5 'pytest-django==3.10.0' 'django>=2.0,<2.1'
 TEST_DATADOG_DJANGO_MIGRATION=1 Python 3.5 'pytest-django==3.10.0' 'django>=2.1,<2.2'
 TEST_DATADOG_DJANGO_MIGRATION=1 Python 3.5 'pytest-django==3.10.0' 'django>=2.2,<2.3'
 TEST_DATADOG_DJANGO_MIGRATION=1 Python 3.6 'pytest-django==3.10.0' 'django>=2.0,<2.1'
 TEST_DATADOG_DJANGO_MIGRATION=1 Python 3.6 'pytest-django==3.10.0' 'django>=2.1,<2.2'
 TEST_DATADOG_DJANGO_MIGRATION=1 Python 3.6 'pytest-django==3.10.0' 'django>=2.2,<2.3'
 TEST_DATADOG_DJANGO_MIGRATION=1 Python 3.6 'pytest-django==3.10.0' 'django>=3.0,<3.1'
 TEST_DATADOG_DJANGO_MIGRATION=1 Python 3.6 'pytest-django==3.10.0' 'django'
 TEST_DATADOG_DJANGO_MIGRATION=1 Python 3.7 'pytest-django==3.10.0' 'django>=2.0,<2.1'
 TEST_DATADOG_DJANGO_MIGRATION=1 Python 3.7 'pytest-django==3.10.0' 'django>=2.1,<2.2'
 TEST_DATADOG_DJANGO_MIGRATION=1 Python 3.7 'pytest-django==3.10.0' 'django>=2.2,<2.3'
 TEST_DATADOG_DJANGO_MIGRATION=1 Python 3.7 'pytest-django==3.10.0' 'django>=3.0,<3.1'
 TEST_DATADOG_DJANGO_MIGRATION=1 Python 3.7 'pytest-django==3.10.0' 'django'
 TEST_DATADOG_DJANGO_MIGRATION=1 Python 3.8 'pytest-django==3.10.0' 'django>=2.0,<2.1'
 TEST_DATADOG_DJANGO_MIGRATION=1 Python 3.8 'pytest-django==3.10.0' 'django>=2.1,<2.2'
 TEST_DATADOG_DJANGO_MIGRATION=1 Python 3.8 'pytest-django==3.10.0' 'django>=2.2,<2.3'
 TEST_DATADOG_DJANGO_MIGRATION=1 Python 3.8 'pytest-django==3.10.0' 'django>=3.0,<3.1'
 TEST_DATADOG_DJANGO_MIGRATION=1 Python 3.8 'pytest-django==3.10.0' 'django'
djangorestframework:
  Python 2.7 'django==1.11' 'djangorestframework>=3.4,<3.5' 'pytest-django==3.10.0'
  Python 2.7 'django==1.11' 'djangorestframework>=3.7,<3.8' 'pytest-django==3.10.0'
  Python 3.5 'django==1.11' 'djangorestframework>=3.4,<3.5' 'pytest-django==3.10.0'
  Python 3.5 'django==1.11' 'djangorestframework>=3.7,<3.8' 'pytest-django==3.10.0'
  Python 3.6 'django==1.11' 'djangorestframework>=3.4,<3.5' 'pytest-django==3.10.0'
  Python 3.6 'django==1.11' 'djangorestframework>=3.7,<3.8' 'pytest-django==3.10.0'
  Python 3.5 'django>=2.2,<2.3' 'djangorestframework>=3.8,<3.9' 'pytest-django==3.10.0'
  Python 3.5 'django>=2.2,<2.3' 'djangorestframework>=3.9,<3.10' 'pytest-django==3.10.0'
  Python 3.5 'django>=2.2,<2.3' 'djangorestframework' 'pytest-django==3.10.0'
  Python 3.6 'django>=2.2,<2.3' 'djangorestframework>=3.8,<3.9' 'pytest-django==3.10.0'
  Python 3.6 'django>=2.2,<2.3' 'djangorestframework>=3.9,<3.10' 'pytest-django==3.10.0'
  Python 3.6 'django>=2.2,<2.3' 'djangorestframework' 'pytest-django==3.10.0'
  Python 3.7 'django>=2.2,<2.3' 'djangorestframework>=3.8,<3.9' 'pytest-django==3.10.0'
  Python 3.7 'django>=2.2,<2.3' 'djangorestframework>=3.9,<3.10' 'pytest-django==3.10.0'
  Python 3.7 'django>=2.2,<2.3' 'djangorestframework' 'pytest-django==3.10.0'
  Python 3.6 'django>=3.0,<3.1' 'djangorestframework>=3.10,<3.11' 'pytest-django==3.10.0'
  Python 3.7 'django>=3.0,<3.1' 'djangorestframework>=3.10,<3.11' 'pytest-django==3.10.0'
  Python 3.8 'django>=3.0,<3.1' 'djangorestframework>=3.10,<3.11' 'pytest-django==3.10.0'
  Python 3.6 'django' 'djangorestframework>=3.11,<3.12' 'pytest-django==3.10.0'
  Python 3.7 'django' 'djangorestframework>=3.11,<3.12' 'pytest-django==3.10.0'
  Python 3.8 'django' 'djangorestframework>=3.11,<3.12' 'pytest-django==3.10.0'

Checklist

  • Entry added to release notes using reno
  • Tests provided; and/or
  • Description of manual testing performed and explanation is included in the code and/or PR.
  • Library documentation is updated.
  • Corp site documentation is updated (link to the PR).

@majorgreys majorgreys requested a review from a team as a code owner October 30, 2020 03:35
brettlangdon
brettlangdon reviewed Oct 30, 2020
View reviewed changes
Copy link
Member

@brettlangdon brettlangdon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if there is a way to validate that the Riotfile was updated to run the same scenarios as tox?

like tox -l | grep django compared against riot list ?

.circleci/config.yml Outdated
command: docker-compose logs -f
background: true
- run:
command: "./scripts/ddtest riot -v run -s 'djangorestframework'"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use something like run_test for docker builds to abstract this part away?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if there is a way to validate that the Riotfile was updated to run the same scenarios as tox?

Updated PR with listing of 69 cases across riot and tox.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use something like run_test for docker builds to abstract this part away?

Yes, definitely think that makes sense but decided to leave that for the next integration we migrate, but can update it here if necessary.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hehe, or do a starting PR to abstract the docker specific stuff.... then this PR for django :p

@majorgreys majorgreys added the changelog/no-changelog A changelog entry is not required for this PR. label Oct 30, 2020
@brettlangdon
Copy link
Member

List of tox envs (=69):

I count 68

@majorgreys majorgreys force-pushed the majorgreys/django-riot branch from 5dcbd92 to d449065 Compare November 6, 2020 03:38
@majorgreys majorgreys force-pushed the majorgreys/django-riot branch from d449065 to 13d3b9c Compare November 6, 2020 03:58
@majorgreys majorgreys force-pushed the majorgreys/django-riot branch from 548a36e to 4b8bd9f Compare November 10, 2020 15:40
@majorgreys majorgreys force-pushed the majorgreys/django-riot branch from 4b8bd9f to f6ece0c Compare November 10, 2020 15:46
Kyle-Verhoog
Kyle-Verhoog approved these changes Nov 10, 2020
View reviewed changes
Copy link
Member

@Kyle-Verhoog Kyle-Verhoog left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not the most elegant setup in the circleci config but imo it's readable which is what matters

@majorgreys majorgreys merged commit 787454d into master Nov 10, 2020
@majorgreys majorgreys deleted the majorgreys/django-riot branch November 10, 2020 18:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@brettlangdon brettlangdon brettlangdon left review comments

@Kyle-Verhoog Kyle-Verhoog Kyle-Verhoog approved these changes

Assignees
No one assigned
Labels
changelog/no-changelog A changelog entry is not required for this PR.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@majorgreys @brettlangdon @Kyle-Verhoog