x/pkgsite: search ranking ought to prioritize more recent major module versions

[cespare]

What is the URL of the page with the issue?

https://pkg.go.dev/search

What is your user agent?

N/A

Screenshot

For example, search for "xxhash":

https://pkg.go.dev/search?q=xxhash

What did you do?

Searched for "xxhash"

What did you expect to see?

The v2 module should be ranked above the old v1 module even if the v1 module has more importers or otherwise ranks higher according to the heuristics pkg.go.dev uses.

Even better, IMO, the v1 wouldn't even show up as a separate search result. The module would appear once and if you visit the page you would see the v2 by default with a way to browse the older module versions.

What did you see instead?

The v1 module appears as the first search result, encouraging new users to use an old version of my module rather than the new one.

[myitcv]

Related #36806 (comment)

[jba]

I'm uncomfortable showing v2 when v1 is more popular. Perhaps v2 is a misfire, and v1 is the better module (see New Coke).

I do think we should group all versions of a module together. Perhaps we can sort that subgroup by version, but mark the most popular in some way (or vice versa—sort by popularity and mark the latest—which I think I like better).

[cespare]

@jba or maybe the reason that v1 is more "popular" is because most people who go looking for the package end up using v1 due to the poor discoverability of later major versions.

As both a consumer and maintainer of open source projects, I prefer nudging toward later major versions. Any popular module that introduces a new major version will go through a long period where more people are using the older version. That doesn't mean that new users should prefer the older version. (And nudging toward "most used" is self-reinforcing: it might be hard for a newer version to catch up if the old version is still adding new users this way.)

[rogpeppe]

I agree with @cespare: I think the latest non-prerelease-tagged major version should always be the main result. If it's not, then no-one will ever be likely to move to a new major version even when there's a strong need to do so, because the old version will always take precedence. Older versions will almost always be more popular, just because they've been around for longer. The author of the package is publishing a new major version specifically because they want people to change to use it.

However, it's reasonable that the most popular version could also have some prominence, because compatibility with other dependencies can also be a consideration.

Here's a screenshot of some recent results:

It's clear that the major version plays no role at all in sorting the results, not even as a secondary sort key after popularity.
In fact, because of the nature of the particular package above, it's likely that almost all the uses are in closed source projects, so popularity is not a good guide.

[mvdan]

Perhaps v2 is a misfire

If a new major version was a mistake that should be ignored, wouldn't those versions be retracted?

[julieqiu]

This will be addressed #47321 (specifically #47320 and #47839), so closing this issue.