Skip to content

Run code coverage for extensions package #4243

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 17, 2020
Merged

Run code coverage for extensions package #4243

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c78d3c4
run code coverage for extensions package
Jul 17, 2020
9b76370
reasonable coverage pct
Jul 17, 2020
1b6eddd
Merge remote-tracking branch 'origin/master' into MLA-1074-ext-coverage
Jul 17, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 28 additions & 20 deletions .yamato/com.unity.ml-agents-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,21 +1,16 @@
test_editors:
- version: 2018.4
# 2018.4 doesn't support code-coverage
coverageOptions:
minCoveragePct: 0
enableCodeCoverage: !!bool false
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See caveat in https://internaldocs.hq.unity3d.com/yamato/yaml/templating/#metadata

- version: 2019.3
coverageOptions: --enable-code-coverage --code-coverage-options 'generateHtmlReport;assemblyFilters:+Unity.ML-Agents'
minCoveragePct: 72
enableCodeCoverage: !!bool true
- version: 2020.1
coverageOptions: --enable-code-coverage --code-coverage-options 'generateHtmlReport;assemblyFilters:+Unity.ML-Agents'
minCoveragePct: 72
enableCodeCoverage: !!bool true
- version: 2020.2
coverageOptions: --enable-code-coverage --code-coverage-options 'generateHtmlReport;assemblyFilters:+Unity.ML-Agents'
minCoveragePct: 72
enableCodeCoverage: !!bool true
trunk_editor:
- version: trunk
coverageOptions: --enable-code-coverage --code-coverage-options 'generateHtmlReport;assemblyFilters:+Unity.ML-Agents'
minCoveragePct: 72
enableCodeCoverage: !!bool true
test_platforms:
- name: win
type: Unity::VM
Expand All @@ -31,7 +26,11 @@ test_platforms:
flavor: b1.medium
packages:
- name: com.unity.ml-agents
assembly: Unity.ML-Agents
minCoveragePct: 72
- name: com.unity.ml-agents.extensions
assembly: Unity.ML-Agents.Extensions
minCoveragePct: 75
---

all_package_tests:
Expand All @@ -57,6 +56,12 @@ all_package_tests:
{% for package in packages %}
{% for editor in test_editors %}
{% for platform in test_platforms %}

{% if editor.enableCodeCoverage %}
{% capture coverageOptions %} --enable-code-coverage --code-coverage-options 'generateHtmlReport;assemblyFilters:+{{ package.assembly }}'{% endcapture %}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No string interpolation in liquid. See https://stackoverflow.com/questions/48074046/how-to-interpolate-in-liquid-shopify and https://github.com/Shopify/liquid/wiki/Liquid-for-Designers#variable-assignment

{% else %}
{% assign coverageOptions = "" %}
{% endif %}
test_{{ package.name }}_{{ platform.name }}_{{ editor.version }}:
name : {{ package.name }} test {{ editor.version }} on {{ platform.name }}
agent:
Expand All @@ -65,11 +70,9 @@ test_{{ package.name }}_{{ platform.name }}_{{ editor.version }}:
flavor: {{ platform.flavor}}
commands:
- npm install upm-ci-utils@stable -g --registry https://artifactory.prd.cds.internal.unity3d.com/artifactory/api/npm/upm-npm
- upm-ci project test -u {{ editor.version }} --project-path Project --package-filter {{ package.name }} {{ editor.coverageOptions }}

{% if package.name == "com.unity.ml-agents" %}
# TODO get coverage tests running for extensions too
- python3 ml-agents/tests/yamato/check_coverage_percent.py upm-ci~/test-results/ {{ editor.minCoveragePct }}
- upm-ci project test -u {{ editor.version }} --project-path Project --package-filter {{ package.name }} {{ coverageOptions }}
{% if editor.enableCodeCoverage %}
- python3 ml-agents/tests/yamato/check_coverage_percent.py upm-ci~/test-results/ {{ package.minCoveragePct }}
{% endif %}
artifacts:
logs:
Expand Down Expand Up @@ -97,20 +100,25 @@ test_{{ package.name }}_{{ platform.name }}_{{ editor.version }}:
{% for package in packages %}
{% for editor in trunk_editor %}
{% for platform in test_platforms %}

{% if editor.enableCodeCoverage %}
{% capture coverageOptions %} --enable-code-coverage --code-coverage-options 'generateHtmlReport;assemblyFilters:+{{ package.assembly }}'{% endcapture %}
{% else %}
{% assign coverageOptions = "" %}
{% endif %}
test_{{ package.name }}_{{ platform.name }}_trunk:
name : {{ package.name }} test {{ editor.version }} on {{ platform.name }}
agent:
type: {{ platform.type }}
image: {{ platform.image }}
flavor: {{ platform.flavor}}
commands:
- python -m pip install unity-downloader-cli --extra-index-url https://artifactory.eu-cph-1.unityops.net/api/pypi/common-python/simple
- python -m pip install unity-downloader-cli --extra-index-url https://artifactory.prd.it.unity3d.com/api/pypi/unity-pypi-local/simple
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not directly relevant to coverage, but we should be using the new URL

- unity-downloader-cli -u trunk -c editor --wait --fast
- npm install upm-ci-utils@stable -g --registry https://artifactory.prd.cds.internal.unity3d.com/artifactory/api/npm/upm-npm
- upm-ci project test -u {{ editor.version }} --project-path Project --package-filter {{ package.name }} {{ editor.coverageOptions }}
{% if package.name == "com.unity.ml-agents" %}
# TODO get coverage tests running for extensions too
- python3 ml-agents/tests/yamato/check_coverage_percent.py upm-ci~/test-results/ {{ editor.minCoveragePct }}
- upm-ci project test -u {{ editor.version }} --project-path Project --package-filter {{ package.name }} {{ coverageOptions }}
{% if editor.enableCodeCoverage %}
- python3 ml-agents/tests/yamato/check_coverage_percent.py upm-ci~/test-results/ {{ package.minCoveragePct }}
{% endif %}
artifacts:
logs:
Expand Down