devcontainer features: Dependency Management

[joshspicer]

How do we handle an inevitable dependency graph for features that depend on other features.

Currently we have no deterministic way to install dev container ‘features’, nor do we have any mechanism to indicate dependencies.

The Codespaces team has been looking to author more features and hope to swap over our primary definition to be a composition of ‘features’. Recently we’ve run into instances where it was a problem that feature installation order couldn’t be defined.

Example: Installing two features, where the installation of A depends on installation of B (JupyterLabs cannot be installed without a particular version of python present).

It is important to explore how such dependency relationships are handled moving forward. Since it’s a complicated problem, this may even lead us to wanted to investigate pivoting a bit, utilizing an existing package manager like npm to create the directed dependency graph for us?

The concept of ‘remote features’ is quite different than the idea of packaging of a single set of features, so I want to discuss this further so we have some plan moving forward.

---

Perhaps someone tangential, but we'll also want this dependency graph to permit multiple "editions" of a feature to be installed:

Example: In the Codespaces base image, we'll want to install node 14 and node 16. How do we specify "installing a feature twice", which one ends up on the $PATH, etc...

[joshspicer]

As a start, I propose we implement a mechanism to define dependencies between features within the same 'feature set'.

We would introduce a dependsOn attribute to the devcontainer-features.json spec, accepting an array of feature id from the same set. Any features indicated here should be installed before the given feature. Note that this will require a change in how flexible we allow the install.sh file to be (ie: additional authoring rules)

features : [
	{
			"id": "jupyter-labs",
			"name": "Jupyter Labs",
			"entrypoint": "/usr/local/share/labs-init.sh",
                        "dependsOn": [
                                "python"
                        ]
                      // ...
	},
	{
			"id": "python",
			"name": "Python 3",
                        "dependsOn": [ ]
                      // ...
	},

]

To actually enforce the order, we may want to copy what vscode-dev-containers does, and actively write the file in such a way that we can enforce order. We then have a default install.sh to execute the scripts, and each install script for each individual feature MUST be kept separate (for example, like in buildpack).

• https://github.com/microsoft/vscode-dev-containers/blob/main/script-library/container-features/src/install.sh
• https://github.com/microsoft/vscode-dev-containers/blob/main/script-library/container-features/src/feature-scripts.env

As @chrmarti pointed out, In the future, we could allow a feature to depend on a feature from another collection by the first collection including a versioned link to the second collection. That would again leave the first collection in control and might have security advantages (e.g., we could add a checksum to the link).

[Chuxel]

Closing this out since I think this has been factored into the current dev container features proposal.

---