x/website: https://golang.org/dl/?mode=json&include=all to https://golang.org/dl/releases.json

[gibfahn]

The releases json file is used by a bunch of tools like rules_go to get a list of go releases.

The current URL works fine for many cases, but trips up mirror sites, for example if you use Artifactory to cache the URL it won't refresh the cache as it doesn't understand the URL format.

Given the comments in the original issue (#23746 (comment)):

@bradfitz - sounds great, I'll get around to this during the weekend. Any thoughts on the releases.json URL? Could probably just be /releases

I don't care much. We use ?mode=json for build.golang.org/?mode=json though.

Would it be acceptable to provide this at an alternate URL like the originally proposed https://golang.org/dl/releases.json? Probably makes sense to mirror both the include=all and the normal releases (see #29380).

This would also make things like curl -O work more nicely (as the downloaded name would be releases-all.json rather than ?mode=json&include=all.

Proposal:

Existing	Mirror
https://golang.org/dl/?mode=json	https://golang.org/dl/releases.json
https://golang.org/dl/?mode=json&include=all	https://golang.org/dl/releases-all.json

[cherrymui]

cc @dmitshur

[jayconrod]

Seems reasonable to mirror the URL. I think that just means routing the request to the same code.

This does sound like a bug in Artifactory though. Query strings are valid parts of URLs. They shouldn't be ignored for caching.

[dmitshur]

The proposed scheme has some convenience improvements in specific cases, but I think the bar for introducing and perpetually maintaining a mirror of an API needs to be higher than minor convenience improvements.

but trips up mirror sites

If this can be fixed in those mirror sites (as Jay suggested), that seems like a better location for the fix.

As one example of API designs with similar properties, GitHub API v3 uses JSON responses also without a ".json" suffix in the URL, and accepts query parameters, such as https://github.com/api/repos/golang/go/events?per_page=3.

[gibfahn]

If this can be fixed in those mirror sites (as Jay suggested), that seems like a better location for the fix.

I don't disagree, although in practice I'm certain it's going to be a much longer process to get Artifactory (and likely other software) to change how they do caching than it would be to use a different URL for this server. Plan is to attempt both 😁 .

As one example of API designs with similar properties, GitHub API v3 uses JSON responses also without a ".json" suffix in the URL, and accepts query parameters, such as api.github.com/repos/golang/go/events?per_page=3.

I guess it depends whether one treats this as a file that is occasionally updated, or an API call (or both). Artifactory treats it as the former (probably because caching API calls in general doesn't make so much sense). I was thinking of this like the Node.js equivalent (https://nodejs.org/dist/index.json).

[justfalter]

It's been almost 4 years, and Artifactory hasn't made any changes to the way that their read-through caches work. Requesting https://my-artifactory-server/artifactory/go-dev/dl/?mode=json&include=all is handled by Artifactory as a local directory listing due to the presence of a trailing forward-slash. Artifactory simply won't forward requests whose path ends with a forward-slash, which is unfortunate.

[gibfahn]

Yes this has been and continues to be a pain for our use-cases. Standing up our own artifactory to fix this one issue isn't very feasible, and it's not easy to configure everything to use an alternate download for go. Go is the only language that requires query params to download (or at least the only one we deal with).