Bump vendored gfortran

[h-vetinari]

Trying to revive #111 in the context of scipy/scipy#13347.

The build infra is not easy to get (with jumping back and forth between multibuild & gfortran-install), but I have the impression that this might not even be using GCC 4.8 anymore (ubuntu 18.04 has gcc 7.2) (nevermind, this is done in scipy-CI, as I myself had documented in the toolchain docs 🤦). I added a commit to display the version, let's see.

[h-vetinari]

@matthew-brett
I cannot tell where the gfortran-4.9.0-Mavericks.dmg on that rackcdn-url

GF_LIB_URL=https://3f23b170c54c2533c070-1c8a9b3114517dc5fe17b7c3f8c63a43.ssl.cf2.rackcdn.com

comes from - that page is empty in my browser, but seems to work in CI.

Is this an official build, or an artefact that someone else prepared? If I wanted to move this to a different gfortran version, what would be the necessary steps?

[mattip]

The artifact is stored in the gfortran-install repo, under archives
https://github.com/h-vetinari/gfortran-install/blob/master/archives/gfortran-4.9.0-Mavericks.dmg
If the filename matches, it is used and not downloaded. The code to check this is https://github.com/MacPython/gfortran-install/blob/4148a01d4b0209e4a5933968a0be62f0f3b2c1ca/gfortran_utils.sh#L83

    local out_fname="${ARCHIVE_SDIR}/$fname"
    if [ ! -e "$out_fname" ]; then
        curl -L "${GF_LIB_URL}/$fname" > $out_fname || (echo "Fetch of $out_fname failed"; exit 1)
    fi
    [ -s $out_fname ] || (echo "$out_fname is empty"; exit 24)
    echo "$out_fname"

[h-vetinari]

Ah, I didn't look into /archive, thanks!

Following the link to the gcc page and then to https://github.com/fxcoudert/gfortran-for-macOS/releases?page=2, I'm wondering why we don't take the binary straight from there? The only thing is that there's no Mavericks (i.e. 10.9) build for gfortran 5.x, only for 10.10. Presumably this would mean bumping the minimum OSX version...?

[rgommers]

It's okay to bump to 10_13 now. This is what cibuildwheel does by default (e.g. https://pypi.org/project/scikit-learn/#files), so the very few users on older macOS are out of luck anyway.

[h-vetinari]

It's okay to bump to 10_13 now. This is what cibuildwheel does by default

Cool! 🙃

So do we go for gfortran 5.5 from homebrew or gfortran 6.3 from fxcoudert/gfortran-for-macOS or some other binary?

[rgommers]

So do we go for gfortran 5.5 from homebrew or gfortran 6.3 from fxcoudert/gfortran-for-macOS or some other binary?

Let's do the bigger bump I'd say. Also, why 5.5 from Homebrew and not for example GCC 8?

[h-vetinari]

Let's do the bigger bump I'd say.

Happy to!

Also, why 5.5 from Homebrew and not for example GCC 8?

Well, tbh I don't know how @matthew-brett, @isuruf, @mattip et al. arrived at the setup to store the binary directly in https://github.com/MacPython/gfortran-install, so I'm not sure if it's feasible to get the binaries from somewhere else.

I haven't used homebrew, but the way I read their page for GCC 8, it looks like the oldest supported MacOS is Mojave (so 10.14, not 10.13).

[rgommers]

Well, tbh I don't know how @matthew-brett, @isuruf, @mattip et al. arrived at the setup to store the binary directly in https://github.com/MacPython/gfortran-install, so I'm not sure if it's feasible to get the binaries from somewhere else.

It's been like that for a very long time, pretty sure there's no better reason than convenience and making sure that any random download page we'd point to does not change from under us. Any new binary can be added in the same place.

[isuruf]

Using homebrew is not a good idea. They support only a few macOS versions. Why not go for a conda based gfortran build like we do for arm64 at https://github.com/MacPython/gfortran-install/blob/master/gfortran_utils.sh#L98 ?

[rgommers]

Sure, happy with conda gfortran, whatever works.

[mattip]

Well, tbh I don't know how @matthew-brett, @isuruf, @mattip et al. arrived at the setup to store the binary directly in https://github.com/MacPython/gfortran-install,

If I recall correctly, there were intermittent problems downloading, and it seemed prudent to cache the binary. There are probably better ways to cache these days, but this one works across all CI systems.

[h-vetinari]

Why not go for a conda based gfortran build like we do for arm64 at https://github.com/MacPython/gfortran-install/blob/master/gfortran_utils.sh#L98 ?

That would be amazing, but would be yet more work for you, no? As in, you'd have to package various (/some) x86 versions for https://github.com/isuruf/gcc/releases as well?

[h-vetinari]

I was thinking that it'd be cool to unify the gfortran version between osx-64 & osx-arm64 (the latter on 10.2 currently); just looked at https://github.com/conda-forge/gfortran_impl_osx-64-feedstock/blob/master/recipe/meta.yaml, and we'd still have to build 10.2 for osx-64, but I could make a PR for that.

From your suggestion to do the same as for osx-arm, I take it that you'd be willing to prepare a similar binary as for osx-arm, @isuruf? Just wanted to double-check...

[h-vetinari]

@isuruf, could you comment on how feasible the gfortran-support through conda-forge is (since it looks like that would depend on you personally)?

Another thought that occurred to me is to incorporate the build-instructions from https://github.com/isuruf/gcc/releases into gfortran-install (or another submodule) directly? That way, the need for you to package the binaries separately would hopefully fall away.

[isuruf]

@isuruf, could you comment on how feasible the gfortran-support through conda-forge is (since it looks like that would depend on you personally)?

It doesn't depend on me personally. All the instructions to reproduce are given in that release.

[h-vetinari]

It doesn't depend on me personally. All the instructions to reproduce are given in that release.

The arm-binaries are published to your fork, but yes, the instructions were exactly the point I was getting at with my last question. Do you think it's worthwhile to do this in gfortran-install?

[isuruf]

Do you think it's worthwhile to do this in gfortran-install?

Installing conda might pollute the env. It's best to not have conda when building the wheel.

[isuruf]

Can you please open an issue in gfortran-install? This is not the appropriate place.

[h-vetinari]

Can you please open an issue in gfortran-install? This is not the appropriate place.

I can open an issue anywhere, but if we don't want to pollute the gfortran-install repo (with the conda install), we could presumably create a new repo/submodule to execute the repackaging of the conda-binaries? In that case, this (repo) might actually the most appropriate place after all?

[isuruf]

I can open an issue anywhere, but if we don't want to pollute the gfortran-install repo

I mean when running the CI like here. Not the repo.

[rgommers]

Archiving this repo and no longer using multibuild, so closing this PR. Thanks all!