Request: use semantic versioning

[khinsen]

Semantic versioning is a widely used convention in software development, distribution, and deployment. In spite of a long-lasting discussion about its appropriateness (Google knows where to find it), it is today the default. Projects that consciously decide not to use semantic versioning tend to choose release numbering schemes that make this immediately clear, such as using dates instead of versions.

NumPy is one of rare examples of widely used software that uses a version numbering scheme that looks like semantic versioning but isn't, because breaking changes are regularly introduced with a change only in the minor version number. This practice creates false expectations among software developers, software users, and managers of software distributions.

This is all the more important because NumPy is infrastructure software in the same way as operating systems or compilers. Most people who use NumPy (as developers or software users) get and update NumPy indirectly through software distributions like Anaconda or Debian. Often it is a systems administrator who makes the update decision. Neither the people initiating updates nor the people potentially affected by breaking changes follow the NumPy mailing list, and most of them do not even read the release notes.

I therefore propose that NumPy adopt the semantic versioning conventions for future releases. If there are good reasons for not adopting this convention, NumPy should adopt a release labelling scheme that cannot be mistaken for semantic versioning.

[eric-wieser]

Could numpy be considered to be using semantic versioning, but with a leading 1?

[rgommers]

Note that almost every core scientific Python project does what NumPy does: remove deprecated code after a couple of release unless that's very disruptive, and only bump the major version number for, well, major things.

Not sure if you're proposing a change to the deprecation policy, or if you think we should be at version 14.0.0 instead of 1.14.09 now.

[khinsen]

The latter: NumPy should be roughly at version 14 by now. But I propose to adopt this convention only for future releases.

BTW: NumPy's predecessor, Numeric, did use semantic versioning and got to version 24 over roughly a decade. I don't know why this was changed in the transition to NumPy.

[njsmith]

My impression is that the vast majority of Python projects do not use semantic versioning. For example, Python itself does not use semantic versioning. (I'm also not aware of any mainstream operating systems or compilers that use semver -- do you have some in mind?) I agree that semver proponents have done a great job of marketing it, leading many developers into thinking that it's a good idea, but AFAICT it's essentially unworkable in the real world for any project larger than left-pad, and I strongly dispute the idea that the semver folks now "own" the traditional MAJOR.MINOR.MICRO format and everyone else has to switch to something else.

Can you give an example of what you mean by a "release labelling scheme that cannot be mistaken for semantic versioning"? Using names instead of numbers? You cite date-based versioning, but the most common scheme for this that I've seen is the one used by e.g. Twisted and PyOpenSSL, which are currently at 17.9.0 and 17.5.0, respectively. Those look like totally plausible semver versions to me...

And can you elaborate on what benefit this would have to users? In this hypothetical future, every release would have some breaking changes that are irrelevant to the majority of users, just like now. What useful information would we be conveying by bumping the major number every few months? "This probably breaks someone, but probably doesn't break you"? Should we also bump the major version on bugfix releases, given the historical inevitability that a large proportion of them will break at least 1 person's code? Can you give any examples of "software developers, software users, and managers of software distributions" who have actually been confused?

[njsmith]

Note that the mailing list is a more appropriate venue for this discussion, and probably we would have to have a discussion there before actually making any change, but the comments here should be useful in getting a sense of what kind of issues you'd want to address in that discussion.

[khinsen]

@njsmith It seems that the only factual point we disagree on is whether or not semantic versioning is the default assumption today. This requires a clearer definition of the community in which it is (or not) the default. The levels of software management I care about is distribution managament and systems administration, which is where people decide which version is most appropriate in their context.

The informal inquiry that led to me the conclusion that semantic versioning is the default consisted of talking to administrators of scientific computing installations. I also envisaged
a more empirical approach (listing the packages on a recent Debian installation and picking a few of them randomly to investigate their versioning approach), but this turned out to be very difficult, because few projects clearly state if they use semantic versioning or not.

A comment from one systems administrator particularly struck me as relevant: he said that for the purposes of deciding which version to install, any convention other than semantic versioning is useless. Systems administrators can neither explore each package in detail (they lack the time and the competence) nor consult all their users (too many of them). They have to adopt a uniform policy, and this tends to be based on the assumption of semantic versioning. For example, an administrator of a computing cluster told me that he checks with a few "power users" he knows personally before applying an update with a change in the major version number.

As for examples of people who have actually been confused, specifically concerning scientific Python users, I have plenty of them: colleagues at work, people I meet at conferences, people who ask for advice by e-mail, students in my classes. This typically starts with "I know you are a Python expert, can you help me with a problem?" That problem turns out to be a script that works on one computer but not on another. Most of these people don't consider dependency issues at all, but a few did actually compare the version numbers of the two installations, finding only "small differences".

[khinsen]

As @eric-wieser and @rgommers noted, my request is almost synonymous to requesting that the initial "1." be dropped from NumPy versions. In other words, NumPy de facto already uses semantic versioning, even though it is not the result of a policy decision and therefore probably not done rigorously. However, it does suggest that NumPy could adopt semantic versioning with almost no change to the current development workflow.

[njsmith]

A comment from one systems administrator particularly struck me as relevant: he said that for the purposes of deciding which version to install, any convention other than semantic versioning is useless.

Unfortunately, semantic versioning is also useless for this :-(. I don't mean to split hairs or exaggerate; I totally get that it's a real problem. But just because a problem is real doesn't mean that it has a solution. You fundamentally cannot boil down the question "should I upgrade this software?" to a simple mechanical check. It's a fantasy. Projects that use semver regularly make major releases that all their users ought to immediately upgrade to, and regularly make breaking changes in minor releases.

Systems administrators can neither explore each package in detail (they lack the time and the competence) nor consult all their users (too many of them). They have to adopt a uniform policy, and this tends to be based on the assumption of semantic versioning. For example, an administrator of a computing cluster told me that he checks with a few "power users" he knows personally before applying an update with a change in the major version number.

I like this part though :-). I doubt we'll agree about the philosophy of semver, but it's much easier to have a discussion about the concrete effects of different versioning schemes, and which outcome we find most desirable.

I don't think the concept of semver has much to do with this policy -- does the system admin you talked to actually check every project to see if they're using semver? Most projects don't, as you said, it's hard to even tell which ones do. And the policy is the same one that sysadmins have been using since long before semver even existed. I think a better characterization of this policy would be: "follow the project's recommendation about how careful to be with an upgrade", along with the ancient tradition that major releases are "big" and minor releases are "little".

The NumPy project's recommendation is that system administrators should upgrade to new feature releases, so what I take from this anecdote is that our current numbering scheme is accurately communicating what we want it to, and that switching to semver would not...

[khinsen]

@njsmith OK, let's turn away from philosophy and towards practicalities: What is the role of software version numbers in the communication between software developers, system maintainers, and software users?

Again it seems that we have a major difference of opinion here. For you, it's the developers who give instructions to system maintainers and users, and use the version numbers to convey their instructions. For me, every player should decide according to his/her criteria, and the version number should act as means of factual communication at the coarsest level.

Given that NumPy has no security implications, I don't see how and why the NumPy project should give universal recommendations. People and institutions have different needs. That's why we have both ArchLinux and CentOS, with very different updating policies.

@khinsen The oldnumeric package still works perfectly, and people can install it with:

pip install oldnumeric

Perhaps this could be your proposed "stable numpy," where the interface to numpy is restricted to Python/Cython and nothing is ever changed. Of course, writing code with oldnumeric is very arcane, but you can't have it both ways.

[khinsen]

@xoviat True, but that's a different issue. My point here is not software preservation, but communication between the different players in software management.

Question: As a systems administrator (even just on your personal machine), would you expect a package to drop a complete API layer from version 1.8 to version 1.9?

For those who replied "yes", second question: can you name any software other than numpy that ever did this?

BTW, I can assure you that many people were bitten by this, because I got a lot of mails asking me why MMTK stopped working from one day to the next. All these people had done routine updates of their software installations, without expecting any serious consequences.

But dropping oldnumeric was not the worst event in recent NumPy history. That honor goes to changing the copy/view semantics of some operations such as diagonal. Code that returns different results depending on the NumPy version (minor version number change!) is a real nightmare.

[khinsen]

BTW, since hardly anyone knows the story: pip install oldnumeric works since two days ago, because @xoviat prepared this add-on package and put it on PyPI. Thanks a lot!

[eric-wieser]

would you expect a package to drop a complete API layer from version 1.8 to version 1.9?

Which layer are you referring to?

[rgommers]

can you name any software other than numpy that ever did this?

SciPy dropped weave and maxentropy packages, pandas breaks major features regularly. I'm sure there are many more prominent examples. EDIT: Python itself for example, see https://docs.python.org/3/whatsnew/3.6.html#removed

BTW, I can assure you that many people were bitten by this, because I got a lot of mails asking me why MMTK stopped working from one day to the next. All these people had done routine updates of their software installations, without expecting any serious consequences.

That change was about 10 years in the making, and there is no way that a different versioning scheme would have made a difference here.

Dropping deprecated features is a tradeoff between breaking a small fraction of (older) code, and keeping the codebase easy to maintain. Overall, if we're erring, then we're likely doing that on the being conservative side. As someone who also has had to deal with many years old large corporate code bases that use numpy I feel your pain, but you're arguing for something that is absolutely not a solution (and in general there is no full solution; educating users about things like pinning versions and checking for deprecation warnings is the best we can do).

[rgommers]

Which layer are you referring to?

numeric/numarray support I assume