Test coverage CI

[jrbourbeau]

To see if we're below 100% on master

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• Docs build locally (e.g., run tox -e docs)
• AppVeyor and Travis CI passes
• Test coverage is 100% (Coveralls passes)

[jakirkham]

If you look at the Coveralls report, it lists files a few times. Some of these note 100% and some don't. Though the ones that don't typically have lines intentionally excluded (workarounds for old Python or NumPy versions). The cumulative result is 100% coverage. However it is curious that Coveralls no longer reports this correctly. I'm guessing they changed something, which is making cumulative coverage not work correctly. Not sure how though. Maybe this helps give you some context. 🙂

[jakirkham]

This seems interesting.

Coverage.py warning: Couldn't read data from '/home/travis/build/zarr-developers/zarr-python/.coverage.travis-job-b9c4d5f1-3db6-48d9-b9c7-def15fa379e3.4893.462486': CoverageException: Doesn't seem to be a coverage.py data file
Coverage.py warning: Couldn't read data from '/home/travis/build/zarr-developers/zarr-python/.coverage.travis-job-b9c4d5f1-3db6-48d9-b9c7-def15fa379e3.4894.722251': CoverageException: Doesn't seem to be a coverage.py data file

ref: https://travis-ci.org/zarr-developers/zarr-python/jobs/610469624#L2314-L2315

[jrbourbeau]

Thanks for the extra info, that does help provide more context. Mostly, I was surprised to see over in #503 two lines (here and here) which are seemingly unrelated to the changes in the PR become uncovered

[jakirkham]

Yeah we might want to come up with a special coverage selector for NumPy version similar to what has been done with Python previously.

zarr-python/tox.ini

Lines 14 to 15 in c647de1

	py35,py36,py37: PY_MAJOR_VERSION = py3
	py27: PY_MAJOR_VERSION = py2

zarr-python/.coveragerc

Line 7 in c647de1

pragma: ${PY_MAJOR_VERSION} no cover

[jakirkham]

Toggling for CI

[jakirkham]

Toggling for CI.

[mzjp2]

For some reason, coveralls just isn't being communicated with, but I can't tell why. Our .travis.yml is the same as always 🤷‍♂

[jakirkham]

FWIW after poking at this problem occasionally over the past few months, I'm about as confused as you are.

Maybe we need to use the travis-pro configuration?

[mzjp2]

Just FYI, I've removed the continuous-integration/travis-ci status check and enabled the other other one, based off it being depracated: https://travis-ci.community/t/known-issue-travis-ci-reports-expected-waiting-for-status-to-be-reported-on-the-github-status-api-but-the-status-never-arrives/1154

[mzjp2]

Other people with the same problem: lemurheavy/coveralls-public#1408

[jakirkham]

Some work to fix Coveralls in PR ( #541 ).

[jrbourbeau]

Closing this PR in favor of #541. Thanks for working on this @mzjp2!