Publish binary releases for Windows

[augi]

It would be great to have binary releases also for Windows.

Related to #109

[sbarzowski]

There are two practical problems with that:

• We currently use Travis for binary releases, which doesn't support Windows.
• None of the maintainers use Windows.

That said, I would really like to see it, especially since problems with compilation on Windows (either manually or through pip) surface from time to time and we are not currently very well equipped to help with that.

[mckunda]

• We currently use Travis for binary releases, which doesn't support Windows.

I would like to correct you on that. Travis actually has support for Windows . I'm not sure about deployment here, but i had experience building with CMake on Windows. It was not easy enough to correctly setup the env variables, though. Here's what I eventually ended up with.

Should I try something similar here?

[sbarzowski]

Thanks, that's good to know.

Should I try something similar here?
That would be great!

[mckunda]

Okay, so, given that Windows has no make, would this part of the build script be enough? (Of course I'll add the script for bundling as well)

- python setup.py build
- python setup.py test
- cmake . -Bbuild && cmake --build build --target run_tests

Also, which version of python should be used?

[sbarzowski]

The most important part is building the actual binary. For Linux and Mac it happens at the make dist stage. I think that something like this should work:

cmake --build build --target jsonnet
cmake --build build --target jsonnetfmt

Then it needs to be packaged (zipped I guess).

The stuff you posted is there for testing, I think. I don't think there is any effect of the Python part of the resultant archive. If it doesn't cause problems it's good to preserve this for Windows, too. Python 3 should be used in such case.

[mcovalt]

I built a package with binary releases for all major platforms (Windows/Linux/macOS).

pip install jsonnetbin

I really only changed setup.py to not use make and just use the built-in setuptools to build the required libraries. I'm not sure how well it works. It should probably be tested against the test_suite. Seems to work so far, though.

I used GitHub Actions with cibuildwheel to build the wheels. Should be easy enough to replicate to Travis CI.

[AndreSteenveld]

Having something native on windows would be great as I ran in to the same problem here. Installing python on my windows machine unfortunately is also not an option. I do have docker running and I've created a small bash script which allows me to pretend that I have jsonnet installed locally 😅. This is only really usable is performance is something you don't care about.

#! /usr/bin/env bash

#
# Clearly access to a docker host is required to make this work... For a windows setup
# we will also be assuming that all the relevant drives (C, D, whatever) are mounted on
# the docker host under "/c", "/d", etc.
#

docker pull --quiet mexisme/jsonnet:latest > /dev/null

( winpty -Xallow-non-tty docker run --rm --tty \
        --env TERM=xterm \
        --volume "//c://c" \
        --workdir "/$( cygpath --unix --absolute $PWD )" \
        mexisme/jsonnet:latest "//jsonnet" "$@"
) | sed -e 's/\x1b\[[^@-~]*[@-~]//g'

# Because winpty is a little finicky with how the control characters are forwarded
# we're going to strip them out. This regex was taken from: https://stackoverflow.com/a/24005600/95019

[zeno17]

I built a package with binary releases for all major platforms (Windows/Linux/macOS).

pip install jsonnetbin

I really only changed setup.py to not use make and just use the built-in setuptools to build the required libraries. I'm not sure how well it works. It should probably be tested against the test_suite. Seems to work so far, though.

I used GitHub Actions with cibuildwheel to build the wheels. Should be easy enough to replicate to Travis CI.

Is there any way I could use this to build the normal jsonnet or trick a conda installation such that it uses jsonnetbin? I am trying to use a library which has jsonnet as its dependency

[sbarzowski]

FWIW I'm open to PRs changing the official jsonnet package to stop depending on Makefile for the Python extension (and use a similar approach as jsonnetbin).

[johnbartholomew]

Ok, we have a few relevant build outputs now:

• The Python package publishing step in the release now includes making prebuilt Windows wheels (and various Linux platforms) which are pushed to PyPI (https://pypi.org/project/jsonnet/0.21.0rc1/#files)
• Releases of go-jsonnet, which is more recommended if you want the command line tool, have prebuilt Windows binaries attached to the GitHub release (https://github.com/google/go-jsonnet/releases/tag/v0.21.0-rc1)

Also, the Python package build (for C++ jsonnet) no longer (as of #1137) relies on make or od tools, so it's little more widely compatible.

We're still missing prebuilt binaries for Windows for C++ jsonnet CLI, but I think that is less important (go-jsonnet should be fine; it's arguably better anyway). We're also missing Windows wheels for the gojsonnet Python package - that would be good to add.

I will optimistically say the heart of this (Windows Python wheels, and - for go-jsonnet - Windows CLI binaries) is done now.