Phase out fpm-bootstrap

[milancurcic]

With #246 merged (great work @LKedward!) a user that arrives to fortran-lang/fpm can easily:

• Download and run a binary for their platform
• Download and run the same binary to bootstrap fpm from source

fpm thus does not anymore need the Haskell fpm for bootstrapping. However, the README.md still asks the user to install Haskell for building from source. The correct instructions should now be to first download the appropriate fpm binary and then run fpm build in the fpm repo directory.

I think we should at this point start to de-emphasize the Haskell implementation in this repo as much as possible. It served its purpose superbly (thank you @everythingfunctional!), but it now can confuse newcomers to the project, which is not helpful at best and harmful at worst (I've seen it more than a few times now). With the ever-shorter attention spans, first impressions matter more than ever.

I propose the following steps to phase out fpm-bootstrap:

1. Update README.md to not require Haskell for building from source.
2. Move the fpm project (contents of https://github.com/fortran-lang/fpm/tree/master/fpm) to the top-level.
3. Move contents of https://github.com/fortran-lang/fpm/tree/master/bootstrap to a separate repo. I suggest either fpm-haskell or fpm-hs instead of fpm-bootstrap, because using the word "bootstrap" here is now misleading. Ditto with the contents of the archive directory (early Rust prototype), though this one I think we should just remove to not confuse anybody. The code will exist in the git history anyway.

Of course, the Haskell fpm should live on if people want to develop or use it.

What do you think?

[LKedward]

I agree; simplifying the process for developers should hopefully increase engagement with the project. I realised during #246 that the Haskell version now needs to move to a separate repo now that we are explicitly versioning the Fortran implementation.

There is still the issue of bootstrapping on non x86_64 platforms, but I don't know how common these are for development.

Presumably we can also delete the 'archive' folder which isn't necessary on git?

[awvwgk]

Sounds like a good plan. Splitting projects on GitHub works usually seamlessly. If we want to preserve the git history we can just clone this repository and push it to the new repo and prune the repositories afterwards. Issues relevant for Haskell fpm can be easily moved within repositories of an organisation from my experience, so nothing gets lost in this transition.

[awvwgk]

There is still the issue of bootstrapping on non x86_64 platforms, but I don't know how common these are for development.

I have some experience compiling and packaging for PowerPC and ARM on conda-forge. For now we don't have to worry, we can either bootstrap a version on PowerPC, ARM, ... later with the Haskell version (if stack is available there and we find a CI provider, drone offers ARM, Travis-CI offers PowerPC and ARM) or just cross-compile using our native x86_64 binary (for OSX ARM which nobody offers on CI yet).

[everythingfunctional]

From the time we decided to (re)implement in Fortran, that had basically always been the plan. If we've reached that point (which it seems we have), then I'm in favor of it.

I will echo @LKedward's concern about other architectures, but my understanding is that bootstrapping for new architectures is usually done via cross-compiler tool chains anyways. So if the Fortran version has sufficient support to allow cross-compiling (or it will be completed very soon), then this isn't really a big issue.

I intend to continue using the Haskell version as a platform for experimentation. If the fortran-lang project does not wish to continue hosting it, I will be happy to keep it going under my personal account. In either case, I hope it becomes less and less necessary over time.

I must say, the amount of support and community contributions this project has received over the last 9 months (I can't believe we only started this project 9 months ago!) has been quite exciting and far exceeded even my wildest expectations. I owe a great deal of gratitude to everyone who has contributed.

[certik]

Thank you Brad, Laurence, Milan, Sebastian and others! Indeed, we are doing really well, I am very happy with the progress so far, and I agree to simply bootstrap using Fortran by default, only keeping the Haskell as a "backup", which won't be the default anymore. Also I think we will soon get #123 working, which will allow to bootstrap fpm from Fortran source using cmake on any architecture.

[certik]

I tested the fpm (Fortran) binary and it works to bootstrap fpm on my Ubuntu 18.04 machine.

I suggest to host the Haskell fpm binaries elsewhere, not at the main fpm repository, because it confuses people:

https://twitter.com/MattChallacombe/status/1336738204452417536

The released binaries are intended for end users, and end users shouldn't need the Haskell version. The Haskell version is for developers, or to bootstrap fpm from scratch (but then you wouldn't use the binary version anyway).

[awvwgk]

I'll go through the issue list and try to cleanup stuff that is already fixed or not relevant anymore. Any help on this is highly appreciated.

[awvwgk]

I pinned this issue since it is important to clarify the relation between the Fortran and Haskell version to avoid further confusion as this milestone issue is now on page two of our issue tracker.

To have some more actionable items to move this forward, the next step would be to split the CI and have Fortran fpm bootstrap itself (related #272) with a separate workflow bootstrapping Fortran fpm with the Haskell version. After some testing on this we found that we relying on some Haskell fpm specifics (see #343 + fix #352), which has to be resolved first.

After #272 is resolved we could plan to split the repositories and have Fortran fpm stand on its own. There is still the question where the Haskell version should go. A repository under @fortran-lang namespace is probably the best idea to get a smooth transition.

[LKedward]

Is the plan here to completely replace Haskell fpm when bootstrapping Fortran fpm, or are we to retain it for building fpm in the CI for binaries? Issues like #388 and the fact that Haskell fpm doesn't support projects with c sources (required for stdlib_os) mean that it would be beneficial to have a clean break from the Haskell implementation, either using existing fpm binaries, a single-file fpm source or third-party build system (make,cmake) to build fpm in the CI.

[awvwgk]

I'm all for bootstrapping Fortran fpm with itself 👍. I was planning to have Fortran fpm write meson build files to allow bootstrapping with different build tools but this is quite a task. If we are fine with adding a third-party build system to this repository I can contribute my personal meson setup I'm using for coverage runs.

[milancurcic]

I agree, the simplest seems to me to just grab an fpm release binary and use that to bootstrap.

[LKedward]

Sounds good! Then it looks like we're on track to complete the phase-out with #389 and then #387 (thanks @awvwgk!).
After which we can create a separate Fortran-Lang repo for the Haskell version.

Separately I'd also like to propose that we mark all fpm-haskell issues as wontfix and close them, based on Brad's comment:

If you really need this fixed in the Haskell version, I am available for contract work to fix it, but I'm not planning on spending much more of my free time (if any) working on the Haskell version since the Fortran version has basically made it obsolete.

Does anyone have an objection to this?

[milancurcic]

Is it possible to migrate a subset of issues to a new repo (fpm-haskell)?

[LKedward]

Is it possible to migrate a subset of issues to a new repo (fpm-haskell)?

Good point, yes this is possible on github 👍

[certik]

Let's move the Haskell version out to a separate repository, and we can still use it for a while at the CI, but eventually I would like to simply bootstrap using the CMake backend for fpm that we will write. (As well as using an fpm binary.)

[everythingfunctional]

I agree with @LKedward that we can move the Haskell version to it's own repo as soon as those issues are done. I think fpm-haskell makes sense as the repo name. And yes, we should move the issues to that repo. I don't know if anyone will ever want to actually work on it, but if it stays usable it might be an easier way to port fpm to new platforms if that ever becomes necessary. But it also might be easier to just set up a cross-compiling toolchain. 🤷

[urbanjost]

Creating a {Cmake,make,meson} packaging seems well within reach but using a simple script to make a single-file source or using external tools to create a Cmake and make(1) interface in the distribution now seem quite simple as an interim solution. I have been using a single-source file( using a simple script and a running fpm(1) installation on a POSIX platform to refresh the file )to bootstrap Fortran fpm on several machines with just git(1) and gfortran(1) in the programming environment for months now. No make, no cmake, no haskell, ... on PCs with Cygwin, mingw, windows bash (or whatever it is being called now) and straight-up MSWindows as well as Linux distros without issue.

Even if future versions incorporate C or other complications arise in producing a single-file source it seems likely a snapshot of the current version will be able to build future versions for a significant amount of time (in which package generation should be available). A single-file bootstrap lets you build the current version even without git or a network with no need for a toolchain. It seems like this refactoring of the co-joined fpm(1) into two could proceed now.

What I would like to see is a bootstrap version available via apt-get(1) or rpm(1) or yum(1) or ... Maybe using the 'other' fpm. Anyone ever try the 'other' fpm to package fpm?

Do we have any measures of the downloads of the binary release of the Fortran version versus a bootstrap version?

[awvwgk]

There is probably a neater way to get this info, but who needs API queries when there is shields.io?

Fortran	Haskell

[awvwgk]

What I would like to see is a bootstrap version available via apt-get(1) or rpm(1) or yum(1) or ... Maybe using the 'other' fpm. Anyone ever try the 'other' fpm to package fpm?

I was hoping for a high-level build system export that is supported out of the box by those package managers... but actually, why not go for the single source file solution. I remember you shared a version of the script earlier in an issue here, but I can't find it anymore.

I would give the conda package manager a shot using just a one source file and a Fortran compiler, can't be that hard, can it?

Edit: Figured it out myself.

[awvwgk]

Anyone want to join on pushing fpm to conda-forge? Comment on conda-forge/staged-recipes#14269 if you want to be a recipe maintainer for fpm. Also we still need a Windows build (either GCC 5 or flang 11 are available).

[urbanjost]

I had never seen shields.io output. Very interesting.

[awvwgk]

I got the workflow split and can now bootstrap Fortran fpm with itself: #419. Once we got this one in, I will open a PR to delete the bootstrap directory. I think now would be the right time to create a new repository for Haskell fpm under the @fortran-lang namespace.

[LKedward]

Great stuff, I've duplicated the repository at https://github.com/fortran-lang/fpm-haskell which we can now prune for just Haskell fpm (fortran-lang/fpm-haskell#1).

[awvwgk]

@LKedward can you adjust the team scopes to give us push access to fpm-haskell?

[LKedward]

You should have access now

[awvwgk]

We made it, Fortran fpm now stands on its own and we can bootstrap without the Haskell version 🎉. Many thanks to all of you for the hard work to make this possible.