Rakudo releases (and probably zef too) need a long-term support strategy #253
Comments
|
Long story short: We should try and designate some version along the year a LTS version, and support it for 5 years or whatever is thought to be reasonable. At the same time, designate some zef version that is guaranteed to work with it also LTS and try and support it for the duration. Maybe also give them some specific designation, 6.d.1 or whatever so that they stand out among the rest. |
|
I don't think this is a viable solution due to the dynamic nature of the ecosystem. I think the way forward is packaging an application and its dependencies together, maybe even with a Raku runtime. I played a little with the idea now that Rakudo is relocatable, but the C dependencies where hard. |
|
Every ecosystem evolves, but you still know which versions are the ones that work with your application. Imagine however that you have your application working with Foo 2.3.1 and Rakudo 2019.10. A bug in Rakudo 2019.09 is discovered, that affects every Rakudo up to the current one. The but is fixed in 2021.01, which includes a series of improvements, some of which are incompatible with Foo 2.3.1. You need to either path Rakudo, or patch Foo so that it works with the new version, of both. Any way, you're in for a bit of yak shaving. Even if packaging everything together is a solution that helps reproducibility, you still need a long-term support strategy for this kind of scenario. |
|
I am looking for a quick win to get you the magical 80% there while keeping the constraints in mind regarding the young age of the ecosystem and the relative small size of the community. I'm looking at things like Perl's App::Fatpacker and Carton or Golang's new module system based around minimal version selection (that took them a very long time to get it right). I think that adding a LTS Rakudo to the mix would require a sizeable time investment in devs, fixing bugs in 2 branches and releasing it. At the speed Rakudo advances it would take long to make this process more difficult with each release. It would be great, sure, but I am not sure if it's viable. Maybe a good discussion candidate for the RSC. Add to the discussion the new role of Rakudo monthly: would be have enough users to test drive the new features/fixes? When talking about modules, it's even more difficult. We can add LTS by example to the travis testing mix, but still there are no warranties people would care about 5 year old releases (for the sake of the argument). And with this said, there will be many cases where the user will have no control which Rakudo version they'll run and stick to whatever the distribution or packagers like me supply. Any way, big thumbs up for thinking about this! |
|
Quite clearly this would need resources. How many, however, will depend on
the bug-fixing strategy. Probably the best would be to prepare patches for
the bugs based on the first release where they appeared, and then apply
them to every LTS release since. Backporting bugfixes might be totally more
difficult. On the other hand, LTS releases will need to be branches,
instead of tags, which is all we have now. I'm not saying we should have
*many* releases. Normally, LTS releases are done once every two year or so.
Still, I really think we should nominate some old release (like the first
6.d, for instance) LTS and support it for a few more years.
|
|
Who would do the work for LTS releases?
Who would pay for this work?
So far our strategy has simply been: do not break existing code. Upgrading
rakudo to a newer version should always work, so sticking to a specific
release should not be necessary.
|
|
I'm leaning on the side of "we aren't big enough for LTS yet". I suspect there are other less work options that will also help users manage Rakudo bugs for their use cases. Promoting Containers being one of them. Finding ways to switch Rakudo versions and keeping the installed modules might be another. Maybe we can come up with more? |
|
Containers are precisely what is bringing me here. I can't seem to be able to install zef in the container environment created in 2017.05. Quite clearly if I had built that container back then it should still be available and working, but that would have implied to create a long-term strategy in the first place. It's a Catch-22 situation here. The same would be my answer to @niner: we can't have the resources to have wide support in the industry unless we devote some resources to strategies that help Raku win wide support in industry.
Except it does not. Again, during the creation of the set of containers, I've bumped into problems with Getopt::Long, which simply stopped working somewhere along the 2017::* releases. Before getting to that, I had to downgrade from the latest release 0.3.4 to 0.2.0 in the 2018.*. Every release introduces new features. It will simply stop working with older releases. It would be a best practice to tag all your releases, and say also which releases are compatible. Which is possible-ish right now, since there are test containers for many Rakudo releases. But not before, so it's simply not done. Baseline is you can't say your module always work, even (relatively) critical ones like this one, unless you're able to test at least the LTS releases. And obviously, as stated in the OP, it's going to be extremely difficult for any company to start using in production unless they have the resources to maintain all the used modules when they start breaking with future releases of Rakudo. So the situation is really Catch-22. We might never get big enough to need a LTS unless we have a LTS strategy. |
|
Except it does not. Again, during the creation of the set of containers,
I've bumped into problems with Getopt::Long, which simply stopped working
somewhere along the 2017::* releases. Before getting to that, I had to
downgrade from the latest release 0.3.4 to 0.2.0 in the 2018.*. Every
release introduces new features. It will simply stop working with older
releases.
Wait, that's two different things. I said, installing a newer Rakudo must not
break your existing code (Rakudo backwards compatibility). You now give
examples of new module versions not running on older Rakudo releases, that's
something else (Rakudo forwards compatibility or module backwards
compatibility).
It would be a best practice to tag all your releases, and say
also which releases are compatible. Which is possible-ish right now, since
there are test containers for many Raku versions. But not before, so it's
simply not done. Baseline is you can't say your module *always work*, even
(relatively) critical ones like this one, unless you're able to test at
least the LTS releases.
If the module version you'd like to use doesn't work on older Rakudo, then
simply upgrade Rakudo to a version where it runs. Since according to our
policy, upgrading Rakudo must not break your code, you should be fine.
|
Containers are the top case for me where you want to pin the runtime and the libraries, what other programming languages call "vendoring". We don't have a standard way to do that. This would fix your problem on a reproducible way, while just updating Rakudo would fix your new incompatible modules problem. (This does not mean that LTS wouldn't be useful, but that the example is not a good illustration.) |
|
WRT modules breaking: I put high hopes in fez (the new ecosystem tony-o is working on). Similar to the CPAN eco, releases will never change and not disappear (in contrast to p6c). Version pinning will then be possible. |
|
We're looking too much at compatibility problems here (me included). That's not the main, critical issue thouth. Security bugs are (or might be). We've had no major security bug so far, but when we do, what's the way forward? Fix it in the next issue, and have everyone upgrade? (we'll probably lose quite a few users in the process). Fix every release so that we don't? (That's going to need quite a few resources, even more than maintain one or two LTS releases). |
|
On Sonntag, 27. Dezember 2020 17:43:57 CET Juan Julián Merelo Guervós wrote:
We've had no major security bug so far, but when we do, what's the way
forward?
Fix it in the next issue, and have everyone upgrade?
Yes.
(we'll probably lose quite a few users in the process).
Why would we?
Fix every release so that we don't?
Unless someone demonstrates why it's necessary, we won't.
What if there's a critical security issue with the Raku
version and you need to upgrade?
Then upgrade?
|
I don't want to derail the issue, so I'll inform on IRC. Does the CPAN-like ecosystem mean losing the single best feature of the Raku ecosyatem: easy publishing through github? |
As said above, theoretically upgrading Rakudo won't break any module. In practice, it does. It's supposed to when you change major version, as it has happened from 6.c to 6.d. Just imagine someone relying on a module that broke in that transition. They would have to patch the module if they want to continue using the latest version of Rakudo. With a LTS strategy, we would have kept patching 6.c up to... I don't know, maybe next year. We're still in time, as we transition to 6.e, to give long term support to a specific release, so that people can rely on it working some years into the future, and so that they can continue using any module they use without fear of breakage when they upgrade (because they won't need to upgrade, at least for some time)
Again, it might break some modules you rely on. There are 300 modules in the ecosystem (probably more if the tests are done now) that have stopped working, probably because they relied on some behavior that became obsolete with some release (6.c or 6.d) or maybe because they were using some ambiguous feature that was disambiguated (in a different direction) or some undocumented feature that simply changed. We can't of course support all undocumented features, or indeed any of them. But LTS support is about stability: you know that the release of Rakudo you rely on will continue working, at least for some time, while you have time to upgrade modules that go with it or develop your own, whatever. Really, I don't see this as a big issue. It's a (relatively small) investment, and it will show commitment on our part to users. |
|
On Sonntag, 27. Dezember 2020 19:33:44 CET Juan Julián Merelo Guervós wrote:
As said above, theoretically upgrading Raku won't break any module. In
practice, it does. It's supposed to when you change major version, as it
has happened from 6.c to 6.d.
No it's not supposed to. But then, you are talking about Raku releases (of
which there were exactly 2 so far: 6.c and 6.d) on the one hand, but raising
examples about breakages caused by Rakudo (of which there were dozens of
releases) on the other hand. That's two very different things.
Of course new Raku versions are not necessarily backwards compatible with
previous versions and they don't have to. That's why you state explicitly
which Raku version your source code expects by saying "use v6.c;" or "use
v6.d;" at the start of your code. This is how we ensure that even breaking
changes don't cause any harm and old code stays functional.
Just imagine someone relying on a module that
broke in that transition. They would have to patch the module if they want
to continue using the latest version of Raku.
No, they wouldn't. The module would still run just fine with the older
language version.
With a LTS strategy, we would
have kept patching 6.c up to... I don't know, maybe next year.
Actually we are patching 6.c and have been doing that since Christmas 2015.
That's what the 6.c-errata branch in roast is. Same is true for 6.d of which
there is a 6.d-errata branch that gets commits when we discover that that
language version has a bug.
We're still
in time, as we transition to 6.e, to give long term support to a specific
release, so that people can rely on it working some years into the future,
and so that they can continue using any module they use without fear of
breakage when they upgrade (because they won't need to upgrade, at least
for some time)
That's exactly what our current policies already should give you.
Again, it might break some modules you rely on. There are 300 modules in the
ecosystem (probably more if the tests are done now) that have stopped
working,
Wait, what?! I've been present (if not more actively involved) for every
single Rakudo release since 2015. At no point has anyone stated that we broke
hundreds of modules? We test every release against all ecosystem modules
precisely to avoid such a scenario. Very, very few modules breakages were
deemed ok, because those modules were already quite broken before the release
and only bugs caused the breakage to stay hidden. Or they were dibbling into
Rakudo's internals.
probably because they relied on some behavior that became obsolete
with some release (6.c or 6.d) or maybe because they were using some
ambiguous feature that was disambiguated (in a different direction) or some
undocumented feature that simply changed. We can't of course support *all*
undocumented features, or indeed any of them. But LTS support is about
stability: you know that the version of Raku you rely on will continue
working, at least for some time, while you have time to upgrade modules
that go with it or develop your own, whatever. Really, I don't see this as
a big issue. It's a (relatively small) investment, and it will show
commitment on our part to users.
We already have large commitment to our users by ensuring that we do not break
their code at all in new releases. How much more commitment can there be?
What are those 300 modules that are supposedly broken by Raku or Rakudo
releases? How could they even be broken by a release of a language
specification? Why didn't we catch the breakage when testing Rakudo releases?
|
|
So there's indeed long-term support for 6.c in the shape of that branch. So when you download 6.c, it's actually the updated branch? Same with 6.d in the future? So there's indeed a LTS strategy, only it's not stated in so many words. Until when will 6.c be supported that way? (Just in case it's not clear enough, that's exactly what I'm asking for) WRT to the questions in the last paragraph, there's this issue #39 which mentions that precise number, although there seems to be 215 for the time being. Breakage can come from all kind of sources, but it's not fixed if it's abandoned. Also, testing extensively with all ecosystem modules is relatively recent, it didn't happen, as far as I can tell, a long time ago. Baseline is, modules will break (due to all kind of legitimate or non-legitimate reasons) and they do. |
|
On Sonntag, 27. Dezember 2020 20:14:00 CET Juan Julián Merelo Guervós wrote:
So there's indeed long-term support for 6.c in the shape of that branch. So
when you download 6.c, it's actually the updated branch? Same with 6.d in
the future? So there's indeed a LTS strategy, only it's not stated in so
many words. Until when will 6.c be supported that way? (Just in case it's
not clear enough, that's exactly what I'm asking for)
Yes, it is. But why is that important? Even if we completely removed the whole
roast repository, nothing at all would change for any user whatsoever. Roast
is for compiler developers and maybe documentation authors. It may have value
as a repository of example code. But in no way is it connected to any end user
code written in Raku.
WRT to the questions in the last paragraph, there's this issue #39 which
mentions that precise number, although there seems to be
[215](https://github.com/Raku/ecosystem-unbitrot/issues) for the time
being.
This list doesn't mean that those modules were broken by a Raku or Rakudo
release (still not sure which of the 2 you actually mean!). That's just a list
of modules that could not be installed on a certain test system. The reasons
could be plenty. Indeed there's not even a requirement for a module to have
ever been installable to enter the list. Or take https://github.com/Raku/
ecosystem-unbitrot/issues/28 or
Raku/ecosystem-unbitrot#54 which show that a module
doesn't even have to be broken in any way to make it into the list. The whole
problem seems to be that the module depends on some native library that wasn't
installed on the test system. Not broken at all and least of all by Rakudo.
Baseline is, modules will break (due to all kind of legitimate or
non-legitimate reasons) and they do.
Please demonstrate this.
The answer to the last question is, I
guess, Rakudo releases started to be tested on all ecosystem modules at one
particular point in time. Before that, all kind of things happened.
Even if we take your assertion without requiring any facts to back it up, your
explanation still only shows that we may have broken stuff years ago. That
still doesn't show that we do so today. So why would we need any new effort
today?
|
|
OK, let me get back to square one here. I probably didn't make my case a good one. So let's clarify stuff here and see if I can gather some support. Let me make clear that a LTS policy would be beneficial for the whole ecosystem, and would probably not suck many resources out of the usual release cycle. First, a long term support policy basically means supporting (via bug fixes) certain releases of a product for a period of time that can be anything at all. LTS support periods can overlap, or they simply can go along with short-term support periods. The main tenet of a LTS policy is to show commitment, by the release community, to certain versions (and clients) so that they can rely on having a stable (let me emphasize this, stable) production platform. As the Wikipedia page indicates,
New releases can be guaranteed to be compatible with old releases. You still have to test them thoroughly before getting them into production, and this will need resources from the company. LTS releases don't. Patch, deploy, it's not going to change at all. Of course some testing will be also needed, but since no new features are introduced, you'll get all the benefit of bug fixes without the uncertainty of new (or dropped) features. Any language I know of, and certainly those that are listed on the Wikipedia, have one. Even things that are not so mission-critical as browsers have one; they are still application platforms and you probably need to rely on them. "Just upgrade" is not really acceptable for someone that has a whole application platform that depends on Raku and suddenly discovers a bug. Then all of a sudden they've got into Rakudo 2020.12, with a slew of new capabilities (and other possible bugs) they need to test all over again, extending their release cycle. And possibly breaking some modules that are used with the existing running version. Because it can be said that, if modules are developed correctly and according to published API, they won't break. Well, I have news for you: they don't. I've had to patch modules all over the ecosystem when I upgrade to a new version of Documentable. Test don't have complete coverage, modules use undocumented features... ... And they also use features that are eventually deprecated. There are almost 300 commmits that use the word "deprecated". Features are deprecated from one release to the next, and rightly so. "undefine" was removed here from a specific place. That might have been used in some published code (although, TBH, I haven't found any)... or in some private code that will stop working. The strategy of testing all modules (or at least the core ones) in every release is a good one. However, that does not account for private code, which might break in new releases and get a breakage for free when all they want is to avoid some bug that has been fixed in the last release. So, again, this is my request. Starting now, create as many LTS releases as we can afford, and keep fixing bugs on them for a reasonable period of time. If we can create only one every four years, that's fair enough. But this would really give Raku reliability and make it more trusted in the minds of companies using them in production. |
|
The first thing coming to mind when considering the LTS is a case where a bug fix for a problem existing in an old LTS version is based on an massive previous change. As the latest example I could point to the fix of #1285 which ended up as a major revamp of coercions. Now, to get it fixed for LTS would either require merging the whole coercion thing or developing a new solution. Or leaving it unfixed altogether which is perhaps the best solution of all. Ok, #1285 wasn't a critical one. But what if it was? My point here is the same @niner made a couple of comments above: we simply don't have resources for LTS. Not until Rakudo gets some kind of sponsorship from a major market player. Until then the best strategy is to:
That's the best we can do so far. |
|
@vrurg again, this is clearly a Catch-22 situation. We're unlikely to get any major sponsorship unless we have a LTS strategy so that some company with enough resources will see this as a stable platform to put major systems into production.
|
|
@JJ please, can you finally make clear if you are talking about Raku or Rakudo? There are so far precicely 2 Raku versions. 6.c and 6.d. A module author only ever has to test exactly one Raku version. The one that the module was developed for. That's it. You declare the langauge version, run your tests, be done. That's certainly not too much to ask. There is no Raku version 2017.06. |
|
Rakudo. I'll edit accordingly. |
JJ
changed the title
You test with which release of what version? Nightly? The first one? Current monthly one? |
|
Since the 6.c version is improved well into the 6.d releases, a good solution for this would be to provide rakudo-level switch that would "switch-off" 6.d features and make the current compiler 6.c only. You can do that now at the file level, but this means that testing for this would involve adding |
|
Since the 6.c version is improved well into the 6.d releases, a good
solution for this would be to provide rakudo-level switch that would
"switch-off" 6.d features and make the current compiler 6.c only.
This would be a solution for what? What would be solved by having such a
switch?
You can
do that now *at the file level*, but this means that testing for this would
involve adding `use v6.c` to every single file (since it's a file-level
pragma), up to and including the modules that are being loaded. A
rakudo-level command line switch would effectively turn the compiler into a
"6.c-only" one, and would allow testing for that version.
What would be the point of forcing Rakudo to run a module that's written for a
6.d compiler with a 6.c compiler?
That would be as if we added a --python command line switch so one can
conveniently run Raku modules using a Python compiler. Why would one want to
do this? What would you expect to happen? Why do you think this would be
useful?
Please, the language version thing is extremely simple: you choose a language
version to develop against, add the corresponding use v6.whatever; statement
to the file, and develop. No further action needed. Just make sure that you
always explicitly state the required language version.
If you have a dependency that requires a certain language version, then that's
it. Your users will need to make sure that their compiler and runtime support
this version. There's no point in testing with anything older than that
version.
|
You actually have the switch. Not for CLI though, but Talking about the corporate non-public code, LTS doesn't guarantee it wouldn't be broken. In particular, a bug fix could break the code which somehow relied on the bug to work. The only way to avoid a situation like this is for the company to fill in an issue and make sure tests are included into the roast. With this in mind all they need is to be always specific about Raku version their code use. I.e. no way they use anything but Maybe I'm mistaken somehow, but overall the versioning support in Raku makes it much less dependent on any kind of LTS than any other language around. I would even be brave enough as to state that language versioning support is the LTS we need. |
|
Quite related to some parts of #365 actually. The moral of the story seems to be that most people (basically everybody apart from the about 5 involved individuals who just know how it's done by heart) have very different understandings about the versioning of Raku (not sure how much Rakudo is involved in that topic; software should probably depend on the language specification rather than Rakudo, anyway). For Rakudo itself: the idea seems to be more or less continuous development, and ever-improving behavior according to the language specification. In this sense, it wouldn't matter what Rakudo version somebody is using - if a bug slips in, the choice is free to upgrade or downgrade to a different compiler version that implements the same language version, not containing the bug. |
Uh oh!
There was an error while loading. Please reload this page.
As more and more Raku applications enter into production, those applications are probably version-locked, so that they're guaranteed to work in a specific version of the language. However, we don't offer support for specific versions, not even Rakudo-Star ones. If some bug or critical security update is created, it's only applied to the next release, not backported to whatever release might be the one used by that (hypothetic) Raku application in production.
The same might be applied to
zef, too. New versions are not guaranteed to work in old Rakudo releases, and old versions of zef are not guaranteed to work with the new repositories (which were changed if I remember correctly after the server meltdown in 2018? 2019?). So there's a catch-22 situation here. You might have a zef version working correctly, only to find out it can't download new versions of modules, even if they could work.The text was updated successfully, but these errors were encountered: