Skip to content

Conversation

urbanjost
Copy link
Contributor

@urbanjost urbanjost commented Feb 19, 2021

This is a reconstituted PR for an alpha version supporting plugins. previous comments are very valuable and should be looked at in closed PR #362. There were issues with rebasing the original PR to merge into the latest master and it was easier to just reintroduce the changes in the code.

@ghost
Copy link

ghost commented Feb 19, 2021

Thank you @urbanjost . It works nicely on Windows 10.

@ghost

This comment has been minimized.

@urbanjost

This comment has been minimized.

@ghost

This comment has been minimized.

@urbanjost

This comment has been minimized.

@ghost

This comment has been minimized.

@urbanjost

This comment has been minimized.

@ghost

This comment has been minimized.

@urbanjost
Copy link
Contributor Author

As @LKedward mentioned in #362, the current file structure prevents fpm(1) from being used as an fpm(1) package, and that would be easy to change.

In addition to the reasons proposed I think for this fpm-NAME subpackage support to be most useful some of the independent apps that could result (such as a snapshot of fpm-search) could be distributed as part of the package with a pure fortran single-file bootstrap version of fpm.

So another example might be the new subcommand could remain with basic features, but variants of fpm-init could exist that like the D language package manager prompt you though building a more elaborate setup; or could let you select from the templates on the web page; ...

Someone might like to make an fpm-license command so you could enter "fpm license MIT" and create a LICENSE file.

So if it were a package and you wanted to upgrade or get a specific version it could be a matter of and fpm update and fpm install; if a new user you get the repository, cd into it and enter 'gfortran bootstrap/fpm.f90 -o fpm;./fpm install;rm ./fpm' or something close to that. That could easily be distributed in a tar file for someone on a stand-alone system that just had git(1) and gfortran(1).

Haskell fpm (h-fpm) can still build packages with fpm.mk and CLI options for loading and linking that Fortran fpm (f-fpm) lacks
and Haskell has a number of standard features that make interacting with the OS easier (until stdlib supplies the equivalent of a POSIX interface for Windows?) and such so I am wondering how they would best co-habitate but I would like to see fpm as an fpm package at least as a beta.

So is it time to have the fledgling leave the nest and have an fpm package for fpm, as proposed?
o could allow for the engine to be used as a library; which could allow for modular development of package builders,
TUI and GUI interfaces, ... if a standard interface is defined (and allowing for calling fpm-NAME as is done with git(1)
and cargo(1) would allow for very transparent use of the subpackages).
o allow for building f-fpm with just git and gfortran installed
o not sure about handling the documentation and examples that are currently "above" the f-fpm fpm/ directory. To support f-fpm and h-fpm should at least the version-specific files be branched?

@milancurcic
Copy link
Member

@urbanjost Are you available for this month's video call? It'd be good to discuss this PR broadly and I'd like to see it go forward. If not, and if you agree, I'd like to present and discuss this PR for you.

@urbanjost
Copy link
Contributor Author

Very unlikely I will make the meeting, so feel free to discuss it.

A few thoughts that might be interesting but very preliminary --

I cloned the fpm repository partly to test fpm-search using an alternate repository but also because I was toying with the idea of making a "plugin" repository with (aware of the catch-22) fpm-search, maybe fpm-single, fpm-man (being what I called fman and fan), a bash shell to get and install a plugin assuming it is an fpm package (which is potentially very limiting) and looking at using "fpm build --show-model" as a way to simulate making plugins model aware to play with the concept. If fpm becomes a package soon that would be easier; but without a firmly defined interface that is just an experment..

So I see a richer support of plugins as potentially quite powerful. I think it could be used to develop features early that could be very system-dependent without polluting the core Fortran fpm functionality (you could have an fpm-cmd for Linux and for WIndows that did the same thing but used gfortran extensions or OS-specific commands more liberally than I would like to see fpm do directly (except via stdlib) and so on.

fpm-search is by far the best example so far; but fpm-single is an indication there is more potential there.

In any case no matter what the direction just allowing fpm-cmd to be called as "fpm cmd" gets the ball rolling. That is somewhat trivial by itself (you could of course just call the command "fpm-cmd") but with some kind of way of being able to query the model
you could picture things like "fpm format" calling a reformatting program on the source files; or "fpm edit PROCEDURE" taking you into your editor on the file containing the routine and so on.

I have a alpha-alpha program called fpm-shell that lets you run the built-in commands in an "fpm shell" where you can call the subcommands without a prefix and plugins too that is an interesting plug-in.

These are just rough ideas to play with at this point, and plugins are supported but do not seem to be a big component of other package managers; which is offsetting -- but supporting a modular model for fpm seems promising so I am behind anyone that wants to advocate for it. Maybe it will just allow for a simple way to prototype functions that will ultimately become core fpm commands but I think that will allow for faster fpm development too so go for it. Wondering if anyone has strong opinions on this before the meeting. That would be interesting to hear.

@LKedward
Copy link
Member

Apologies for the delay with this PR; I am in support of this approach but haven't yet had a chance to look at it in detail and play around with it.

Also, as noted above, fpm needs restructuring before plugins can properly access the model, but there are no fundamental blockers on why this can't be done now. I would advise against trying to parse the output of fpm --show-model to access the model.

@milancurcic
Copy link
Member

@urbanjost we discussed this PR on the call, you can see it here (relevant discussion at about 42:45). Overall there was support for this approach going forward, with some suggestion for improvement, such as not searching for plugin executable in the global PATH, but perhaps only in a specific directory that is not in the PATH.

Copy link
Member

@LKedward LKedward left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @urbanjost, with fpm now residing in the repo root, we can start to support prototyping and tinkering with plugins by using fpm as a dependency.

I have no major concerns with these changes, only a few minor comments.
Am I right in thinking that the which function may return a path even for non-executable files? Not a major issue, but could cause some tricky issues.

The separator function looks like a useful addition to avoid issues on cygwin/msys etc..

There was some discussion in the Monthly call about not searching the path but requiring a plugin manifest of sorts. I actually think the current implementation is fine as a starting point. IMO there shouldn't really be any security concerns here since your system would already be compromised if you can't trust an executable on your path.

@urbanjost
Copy link
Contributor Author

I have a which(3f) function that uses ISO_C_BINDING interfaces to POSIX routines that checks execute permission, but did not have an equivalent for other systems so I simplified which(3f) so it just tests for file existence, which can be done with pure Fortran. I am assuming that stdlib will provide such a function in the future. I started a separate module in my private files called M_OS where I was experimenting with the reliability of separator, the reliability of testing for certain theoretically OS-specific files, environmental variables and other ways to emulate the uname(3f) POSIX function but have not had the time recently to pursue that, put in my own projects I have found that separator is useful of itself, particularly in MSWIndows where old tricks like testing for the directory ./ no longer work as POSIX and DOS filepaths are sometimes both supported. If fpm is assumed to require gfortran there are gfortran extensions that can be used to solve all those issues. Essentially, the M_OS module is just a module whose purpose is to determine OS-specific information (what file separator should I use, what OS am I using (letting me assume certain commands will be available, for example). Considering in the POSIX world I just call uname(1) it is surprisingly complex to just say "where am I" in a really robust way.

So I am aware of the limitations but felt this was sufficient for an initial prototype. A plugin becomes potentially much more powerful if it can use fpm(1) as a package. This prototype is essentially laying some groundwork for exploring the possibilities
of a modular project environment. If a well-defined API is defined for even a few parts of the core pure-Fortran core of fpm then
you could do things like have separate programs for different OSes for other functions but still let the user interface with fpm as an integrated package, for example. As-is, it is really just providing an alias for a command but I think it has intriguing potential for allowing functions such as search, GUI interfaces, and system-dependent functions to develop in parallel with the core functionality. And at this point it is essentially transparent and can be ignored by users of the core functionalities.

Copy link
Member

@awvwgk awvwgk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With two functions to check the OS we could get into a situation where get_os_type() == OS_WINDOWS and separator() == "\" disagree. I would prefer to have one function to handle the OS detection logic with a single point of failure rather than multiple different.

Can we merge the logic of separator with get_os_type to refine our OS detection logic and than use get_os_type in separator to return the correct one?

Copy link
Member

@milancurcic milancurcic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pending others' reviews, I support this going forward.

Copy link
Member

@awvwgk awvwgk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@urbanjost This patch also enables the response file support from M_CLI2 if I have seen this correctly. Can we add a short sentence on this in the help printout as well? Something like:

@file
     Read command-line arguments from file. The arguments read
     are inserted in place of the original @file argument.
     If file does not exist or cannot be read, then the argument
     will be treated literally, and not removed. Each line in file
     is interpreted as a separate argument. The file may itself contain
     @file arguments. Any such argument will be processed recursively.

@milancurcic
Copy link
Member

@urbanjost, I think there is a strong consensus to move this PR forward. Do you need help resolving the conflicts?

@urbanjost
Copy link
Contributor Author

Been gone for a bit.

I cannot use github to resolve the conflicts or to look at them. Everything having to do with the conflicts appears to be grayed out.
Is there a button I am missing somewhere? I want to evolve this into something that can see the fpm model via an API but starting with this is a reasonable first step and should be upward-compatible with any such feature.

@urbanjost urbanjost closed this Jun 1, 2021
@urbanjost urbanjost deleted the plugins-alpha branch June 1, 2021 05:23
@urbanjost urbanjost mentioned this pull request Jun 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants