Build + Support Py39 and Various CI Updates

[evamaxfield]

I was originally trying to get numcodecs for py39 + macOS building as I have a build failing, but, in working on this:

• I saw Try to migrate to Gh-action. #265 (Updating CI / moving from Travis to GitHub Actions)
• I also saw Can't install numcodecs on OSX Big Sur #269 (blosc upgrade)
  • But then saw in zarr dev community call notes that it may be preferred to move from blosc submodule to blosc pip install.
  • And all the tests + wheel builds on MacOS + Py39 so not sure where to fall on this

In addressing the CI changes, I tried to figure out the travis and appveyor prior setups but it was a bit tricky. The one missing thing is coveralls which has a GitHub Action that we could use but didn't know how you wanted to handle that. (Sidenote: other option is to move to codecov which is very easy to setup in my experince)

Reference Issues:
#262
#261
#269

TODO:

• Unit tests and/or doctests in docstrings
• tox -e py39 passes locally
• Docstrings and API docs for any new/modified user-facing classes and functions
• Changes documented in docs/release.rst
• tox -e docs passes locally
• Test coverage to 100% (Coveralls passes)

[joshmoore]

Oh wow. Nice, @JacksonMaxfield. Briefly on the trade-off: @jakirkham suggests that moving to the upstream wheels will be doable and advantageous but will require a number of changes throughout the build system. Once done, it would mean that this repo could be python only. (But no one has yet started on that work) But it looks like you are close so worth moving forward?

If so, moving to codecov is completely valid and has been done elsewhere in the org. For that matter, I could see ignoring leaving it temporarily as a TODO as we figure out the other questions.

[evamaxfield]

But it looks like you are close so worth moving forward?

Sorry, is this saying I should add the blosc wheel support to this PR or saying handling it in the future. I am a bit afraid to touch any of the c / pyx files not going to lie 😅

[Carreau]

Is it really necessary, or can we build wheel only on one of the two linux ?

[evamaxfield]

I don't think both are needed. So I will just replace with ubuntu-latest.

[Carreau]

I've disable making Appveyor mandatory if we move to gh-action so this can be merged even with appveyor failing. I'm +1 on moving to codecov as well.

[Carreau]

Sorry, is this saying I should add the blosc wheel support to this PR or saying handling it in the future. I am a bit afraid to touch any of the c / pyx files not going to lie 😅

I think we can wait for the migration to the blosc wheel, i will likely not need to touch c/pyx beyond deleting them though, but might need minimal API change internally; I think there was a discussion to at least bump the minor version when we do that.

[joshmoore]

Sorry, is this saying I should add the blosc wheel support to this PR or saying handling it in the future

The latter.

I think we can wait for the migration to the blosc wheel,

Exactly.

;) Sorry for being unclear.

[jakirkham]

This looks really good! Thanks for working on this Jackson 😄

Did we want to add Windows to GitHub Actions as well?

[evamaxfield]

Did we want to add Windows to GitHub Actions as well?

That should be pretty easy to do. Just didn't see any windows CI stuff in the repo besides the wheel building. I will add it in later today.

[evamaxfield]

@jakirkham I spoke too soon. It seems that there is semi-known weird numpy + pickle interaction which results in a single one of the tests failing. Specifically the backwards compatibility test on pickles.

StackOverflow explaining the issue. I think it has to do with the "\r\n" / windows carrage return character.

Alright if I skip that test on Windows?

[jakirkham]

Thanks Jackson! 😄

Yeah that sounds like a good idea. Maybe we can mark it as xfail on Windows? Could you please raise an issue as well so we can track fixing that in the future? 🙂

[evamaxfield]

Yeah that sounds like a good idea. Maybe we can mark it as xfail on Windows? Could you please raise an issue as well so we can track fixing that in the future? slightly_smiling_face

Sounds good! Should be good to go w/ the latest Windows CI passing

[jakirkham]

Looks good. Thanks for adding that 🙂

Made a small suggestion to the xfail comment. Otherwise LGTM

[jakirkham]

I've also checked the box to disable AppVeyor builds when appveyor.yml is missing. So hopefully that status check will disappear.

[jakirkham]

Thanks again Jackson! 😄

[evamaxfield]

Thank you all for making this package to begin with 🙂

[evamaxfield]

Hey @jakirkham any chance to bumpversion and release to support py39?

[joshmoore]

Tagged & pushed. 🤞🏼

[evamaxfield]

Last update before I head to bed: my tests now pass w/ Mac Py39 support so hopefully not jinxing it when I say "seems like everything worked" 🙂

[jakirkham]

Should now have wheels and Conda packages.

[joshmoore]

@jakirkham : on that note, it looked like in docs/contributing.rst that I should take the checksum of the local .tar.gz?? Can you explicame a bit?

[jakirkham]

Ah that's outdated. We can drop that. I think that was when we were still using GitHub archives instead of PyPI as the source of conda-forge packages

[jakirkham]

Fixing the doc in PR ( #272 )