[MAJOR] Restructuring the repository, and backwards compatibility concerns

[pradyunsg]

Coming from pypa/get-virtualenv#1

This is an issue for discussing potential areas of concern when we rework bootstrap.pypa.io's deployment assumptions and, therefore, this repository's layout.

[pradyunsg]

I've discussed this on other channels (private chats, IRC, etc) with various folks (including @ewdurbin), summarizing those discussions:

• Should be easy enough to update our invoke task.
  • Would be a good chance to upgrade/switch to nox. :P
• We can't serve redirects from the current bootstrap.pypa.io/get-pip.py since curl does not follow redirects by default.
  • Do we want to serve the same file, or a file containing an additional warning?
• Can we compute digests on the server side, or do we want them to be a part of this repository?
  • The latter seems easier.

[pradyunsg]

@ewdurbin How would we handle the transition between "current" and "next-gen"?

Should I make a commit that adds the next-gen variants in a public/ folder, and then, wait till we switch the deployment over? In a follow up, I'll likely replace the existing scripts with helpful-error-printing-scripts, which we'll keep around for 2 weeks or so.

[ewdurbin]

So is the goal to eventually remove:

• /2.6/get-pip.py
• /3.2/get-pip.py
• /3.3/get-pip.py
• /3.4/get-pip.py

and replace them with

• /pip/2.6/get-pip.py
• /pip/3.2/get-pip.py
• /pip/3.3/get-pip.py
• /pip/3.4/get-pip.py

?

If so, yes the initial step would simply be to add the public directory, which would be updated to serve from bootstrap.pypa.io/pip, symlinking public/get-pip.py so that bootstrap.pypa.io/get-pip.py serves the latest and greatest.

Once that's deployed, you could drop get-pip.py from the root of the repo and adjust the other versions to do as you want.

Eventually when you're ready to fully deprecate the top level version directories, we can remove them from the configuration on the server and they can be removed from this repository as well.

[pradyunsg]

TBH, my goal is to make it so that we don't need to make a PR to psf-salt whenever pip drops support for a Python minor version. I figure that moving the scripts in this repository into a "clean" public directory that can just be dumped into the bootstrap.pypa.io folder, should make it straightforward for us to decouple that.

I'm "meh" on making changes to the server side URLs -- it would need transitioning users though, so if someone wants to do that, we should definitely discuss how we'd go about doing that. :)

[pradyunsg]

FWIW, I've filed #102 and if @ewdurbin is generally positive for making the change I described above (change the psf-salt to "just use" the public directory in this repository), then we can go ahead and:

• merge Add current scripts to public/ directory #102. :)
• make the changes in psf-salt, and let the server update.
• change the non-public scripts in this repository to error out with a message pointing to the new location.
• wait 4 weeks.
• drop the non-public scripts from this repository.

[pradyunsg]

TBH, my goal is to make it so that we don't need to make a PR to psf-salt whenever pip drops support for a Python minor version.

@ewdurbin nudge to ask if this is possible by moving this project's psf-salt to using public/, so that this project can remove the generated scripts in the root of this repo.

[ewdurbin]

I'm all for this approach. The only issue I'm seeing is that the structure of directory "composes" the various projects all together under the root:

ernestd@web0:/srv/bootstrap/www$ sudo tree -a
.
├── 2.6
│   └── get-pip.py -> /srv/bootstrap/pip/2.6/get-pip.py
├── 2.7
│   └── get-pip.py -> /srv/bootstrap/pip/2.7/get-pip.py
├── 3.2
│   └── get-pip.py -> /srv/bootstrap/pip/3.2/get-pip.py
├── 3.3
│   └── get-pip.py -> /srv/bootstrap/pip/3.3/get-pip.py
├── 3.4
│   └── get-pip.py -> /srv/bootstrap/pip/3.4/get-pip.py
├── 3.5
│   └── get-pip.py -> /srv/bootstrap/pip/3.5/get-pip.py
├── bootstrap-buildout.py -> /srv/bootstrap/buildout/bootstrap/bootstrap.py
├── ez_setup.py -> /srv/bootstrap/setuptools/ez_setup.py
├── get-pip.py -> /srv/bootstrap/pip/get-pip.py
├── virtualenv -> /srv/bootstrap/virtualenv/public
└── virtualenv.pyz -> /srv/bootstrap/virtualenv/public/virtualenv.pyz

Thus there would remain some breakage to URLs to make it work, we could leave the existing 2.6, 2.7,...3.4, 3.5 symlinks, or perhaps implement redirects (that wouldn't be followed by default necessarily)... so new (and existing) versioned would be accessible via https://bootstrap.pypa.io/pip/$VERSION/get-pip.py

[ewdurbin]

ah, missed your "error out" step, that works too :-D