Future / proposed spec changes (#56)

bamurtaugh · bamurtaugh · Chuxel · web-flow · commit 1f1903623ed0 · 2022-11-02T11:19:02.000-07:00
* Add parallel execution

* Add feature env var

* Remove proposal word

* Removal

* Add info on object format

* Add id proposal

* Update _implementors/features.md

Co-authored-by: Chuck Lantz &lt;clantz@microsoft.com&gt;

* Make sub-heading

* Add to variable table

* Feedback

* Update features.md

Co-authored-by: bamurtaugh &lt;brmurtau@microsoft.com&gt;
Co-authored-by: Chuck Lantz &lt;clantz@microsoft.com&gt;
diff --git a/_implementors/features.md b/_implementors/features.md
@@ -80,6 +80,69 @@ The options property contains a map of option IDs and their related configuratio
 | `optionId.description` | string | Description for the option. |
 {: .table .table-bordered .table-responsive}
 
+### <a href="#user-env-var" name="user-env-var" class="anchor"> User environment variables </a>
+
+Feature scripts run as the `root` user and sometimes need to know which user account the dev container will be used with.
+
+`_REMOTE_USER` and `_CONTAINER_USER` environment variables are passsed to the Features scripts with `_CONTAINER_USER` being the container's user and `_REMOTE_USER` being the configured `remoteUser`. If no `remoteUser` is configured, `_REMOTE_USER` is set to the same value as `_CONTAINER_USER`.
+
+Additionally, the home folders of the two users are passed to the Feature scripts as `_REMOTE_USER_HOME` and `_CONTAINER_USER_HOME` environment variables.
+
+The container user can be set with `containerUser` in the devcontainer.json and image metadata, `user` in the docker-compose.yml, `USER` in the Dockerfile, and can be passed down from the base image.
+
+### <a href="#dev-container-id" name="dev-container-id" class="anchor"> Dev Container ID </a>
+
+An identifier will be referred to as `${devcontainerId}` in the devcontainer.json and the Feature metadata and that will be replaced with the dev container's id. It should only be used in parts of the configuration and metadata that is not used for building the image because that would otherwise prevent pre-building the image at a time when the dev container's id is not known yet. Excluding boolean, numbers and enum properties the properties supporting `${devcontainerId}` in the Feature metadata are: `entrypoint`, `mounts`, `customizations`.
+
+Implementations can choose how to compute this identifier. They must ensure that it is unique among other dev containers on the same Docker host and that it is stable across rebuilds of dev containers. The identifier must only contain alphanumeric characters. We describe a way to do this below.
+
+#### <a href="#label-based" name="label-based" class="anchor"> Label-based Implementation  </a>
+
+The following assumes that a dev container can be identified among other dev containers on the same Docker host by a set of labels on the container. Implementations may choose to follow this approach.
+
+The identifier is derived from the set of container labels uniquely identifying the dev container. It is up to the implementation to choose these labels. E.g., if the dev container is based on a local folder the label could be named `devcontainer.local_folder` and have the local folder's path as its value.
+
+E.g., the [`ghcr.io/devcontainers/features/docker-in-docker` Feature](https://github.com/devcontainers/features/blob/main/src/docker-in-docker/devcontainer-feature.json) could use the dev container id with:
+
+```jsonc
+{
+    "id": "docker-in-docker",
+    "version": "1.0.4",
+    // ...
+    "mounts": [
+        {
+            "source": "dind-var-lib-docker-${devcontainerId}",
+            "target": "/var/lib/docker",
+            "type": "volume"
+        }
+    ]
+}
+```
+
+#### <a href="#label-based-comp" name="label-based-comp" class="anchor"> Label-based Computation  </a>
+
+- Input the labels as a JSON object with the object's keys being the label names and the object's values being the labels' values.
+	- To ensure implementations get to the same result, the object keys must be sorted and any optional whitespace outside of the keys and values must be removed.
+- Compute a SHA-256 hash from the UTF-8 encoded input string.
+- Use a base-32 encoded representation left-padded with '0' to 52 characters as the result.
+
+JavaScript implementation taking an object with the labels as argument and returning a string as the result:
+
+```js
+const crypto = require('crypto');
+function uniqueIdForLabels(idLabels) {
+	const stringInput = JSON.stringify(idLabels, Object.keys(idLabels).sort()); // sort properties
+	const bufferInput = Buffer.from(stringInput, 'utf-8');
+	const hash = crypto.createHash('sha256')
+		.update(bufferInput)
+		.digest();
+	const uniqueId = BigInt(`0x${hash.toString('hex')}`)
+		.toString(32)
+		.padStart(52, '0');
+	return uniqueId;
+}
+```
+
 ## <a href="#devcontainer-json-properties" name="devcontainer-json-properties" class="anchor"> devcontainer.json properties </a>
 
 Features are referenced in a user's [`devcontainer.json`](../json_reference) under the top level `features` object.
@@ -308,4 +371,3 @@ There are several things to keep in mind for an application that implements Feat
 - Parameters like `capAdd`, `securityOp`  are concatenated.
 - `containerEnv` is added before the feature is executed as `ENV` commands in the Dockerfile.
 - Each Feature script executes as its own layer to aid in caching and rebuilding.
-- There is also a proposal to move properties like `privileged` to `devcontainer.json` for consistency and to reduce reliance on the `runArgs` property. See https://github.com/devcontainers/spec/issues/2.
diff --git a/_implementors/json_reference.md b/_implementors/json_reference.md
@@ -76,12 +76,12 @@ When creating or working with a dev container, you may need different commands t
 
 | Property | Type  | Description |
 |:------------------|:------------|:------------|
-| `initializeCommand` | string,<br>array | A command string or list of command arguments to run on the **host machine** before the container is created. .<br /><br /> ⚠️ The command is run wherever the source code is located on the host. For cloud services, this is in the cloud.<br><br>Note that the array syntax will execute the command without a shell. You can [learn more](#formatting-string-vs-array-properties) about formatting string vs array properties. |
-| `onCreateCommand` 🏷️ | string,<br>array | This command is the first of three (along with `updateContentCommand` and `postCreateCommand`) that finalizes container setup when a dev container is created. It and subsequent commands execute **inside** the container immediately after it has started for the first time.<br /><br> Cloud services can use this command when caching or prebuilding a container. This means that it will not typically have access to user-scoped assets or secrets.<br><br>Note that the array syntax will execute the command without a shell. You can [learn more](#formatting-string-vs-array-properties) about formatting string vs array properties. |
-| `updateContentCommand` 🏷️ | string,<br>array  | This command is the second of three that finalizes container setup when a dev container is created. It executes inside the container after `onCreateCommand` whenever new content is available in the source tree during the creation process.<br><br />It will execute at least once, but cloud services will also periodically execute the command to refresh cached or prebuilt containers. Like cloud services using `onCreateCommand`, it can only take advantage of repository and org scoped secrets or permissions.<br><br>Note that the array syntax will execute the command without a shell. You can [learn more](#formatting-string-vs-array-properties) about formatting string vs array properties. |
-| `postCreateCommand` 🏷️ | string,<br>array | This command is the last of three that finalizes container setup when a dev container is created. It happens after `updateContentCommand` and once the dev container has been assigned to a user for the first time.<br><br />Cloud services can use this command to take advantage of user specific secrets and permissions.<br><br>Note that the array syntax will execute the command without a shell. You can [learn more](#formatting-string-vs-array-properties) about formatting string vs array properties. |
-| `postStartCommand` 🏷️ | string,<br>array | A command to run each time the container is successfully started.<br><br>Note that the array syntax will execute the command without a shell. You can [learn more](#formatting-string-vs-array-properties) about formatting string vs array properties. |
-| `postAttachCommand` 🏷️ | string,<br>array | A command to run each time a tool has successfully attached to the container.<br><br>Note that the array syntax will execute the command without a shell. You can [learn more](#formatting-string-vs-array-properties) about formatting string vs array properties. |
+| `initializeCommand` | string,<br>array,<br>object | A command string or list of command arguments to run on the **host machine** before the container is created. .<br /><br /> ⚠️ The command is run wherever the source code is located on the host. For cloud services, this is in the cloud.<br><br>Note that the array syntax will execute the command without a shell. You can [learn more](#formatting-string-vs-array-properties) about formatting string vs array vs object properties. |
+| `onCreateCommand` 🏷️ | string,<br>array,<br>object | This command is the first of three (along with `updateContentCommand` and `postCreateCommand`) that finalizes container setup when a dev container is created. It and subsequent commands execute **inside** the container immediately after it has started for the first time.<br /><br> Cloud services can use this command when caching or prebuilding a container. This means that it will not typically have access to user-scoped assets or secrets.<br><br>Note that the array syntax will execute the command without a shell. You can [learn more](#formatting-string-vs-array-properties) about formatting string vs array vs object properties. |
+| `updateContentCommand` 🏷️ | string,<br>array,<br>object  | This command is the second of three that finalizes container setup when a dev container is created. It executes inside the container after `onCreateCommand` whenever new content is available in the source tree during the creation process.<br><br />It will execute at least once, but cloud services will also periodically execute the command to refresh cached or prebuilt containers. Like cloud services using `onCreateCommand`, it can only take advantage of repository and org scoped secrets or permissions.<br><br>Note that the array syntax will execute the command without a shell. You can [learn more](#formatting-string-vs-array-properties) about formatting string vs array vs object properties. |
+| `postCreateCommand` 🏷️ | string,<br>array,<br>object | This command is the last of three that finalizes container setup when a dev container is created. It happens after `updateContentCommand` and once the dev container has been assigned to a user for the first time.<br><br />Cloud services can use this command to take advantage of user specific secrets and permissions.<br><br>Note that the array syntax will execute the command without a shell. You can [learn more](#formatting-string-vs-array-properties) about formatting string vs array vs object properties. |
+| `postStartCommand` 🏷️ | string,<br>array,<br>object | A command to run each time the container is successfully started.<br><br>Note that the array syntax will execute the command without a shell. You can [learn more](#formatting-string-vs-array-properties) about formatting string vs array vs object properties. |
+| `postAttachCommand` 🏷️ | string,<br>array,<br>object | A command to run each time a tool has successfully attached to the container.<br><br>Note that the array syntax will execute the command without a shell. You can [learn more](#formatting-string-vs-array-properties) about formatting string vs array vs object properties. |
 | `waitFor` 🏷️ | enum | An enum that specifies the command any tool should wait for before connecting. Defaults to `updateContentCommand`. This allows you to use `onCreateCommand` or `updateContentCommand` for steps that must happen before `devcontainer.json` supporting tools connect while still using `postCreateCommand` for steps that can happen behind the scenes afterwards. |
 {: .table .table-bordered .table-responsive}
 
@@ -117,9 +117,12 @@ The `portsAttributes` and `otherPortsAttributes` properties allow you to map def
 
 The format of certain properties will vary depending on the involvement of a shell.
 
-`postCreateCommand`, `postStartCommand`, `postAttachCommand`, and `initializeCommand` all have an array and a string type, while `runArgs` only has the array type. An array is passed to the OS for execution without going through a shell, whereas a string goes through a shell (it needs to be parsed into command and arguments).
+`postCreateCommand`, `postStartCommand`, `postAttachCommand`, and `initializeCommand` all have 3 types: 
+* Array: Passed to the OS for execution without going through a shell
+* String: Goes through a shell (it needs to be parsed into command and arguments)
+* Object: All lifecycle scripts have been extended to support `object` types to allow for [parallel execution](https://containers.dev/implementors/spec/#parallel-exec)
 
-Using `runArgs` via a typical command line, you'll need single quotes if the shell runs into parameters with spaces. However, these single quotes aren't passed on to the executable. Thus, in your `devcontainer.json`, you'd follow the array format and leave out the single quotes:
+`runArgs` only has the array type. Using `runArgs` via a typical command line, you'll need single quotes if the shell runs into parameters with spaces. However, these single quotes aren't passed on to the executable. Thus, in your `devcontainer.json`, you'd follow the array format and leave out the single quotes:
 
 ```json
 "runArgs": ["--device-cgroup-rule=my rule here"]
@@ -131,7 +134,7 @@ Rather than:
 "runArgs": ["--device-cgroup-rule='my rule here'"]
 ```
 
-We can compare the string and the array versions of `postAttachCommand` as well. You can use the following string format, which will remove the single quotes as part of the shell's parsing:
+We can compare the string, array, and object versions of `postAttachCommand` as well. You can use the following string format, which will remove the single quotes as part of the shell's parsing:
 
 ```json
 "postAttachCommand": "echo foo='bar'"
@@ -143,6 +146,17 @@ By contrast, the array format will keep the single quotes and write them to stan
 "postAttachCommand": ["echo", "foo='bar'"]
 ```
 
+Finally, you may use an object format:
+
+```json
+{
+  "postAttachCommand": {
+    "server": "npm start",
+    "db": ["mysql", "-u", "root", "-p", "my database"]
+  }
+}
+```
+
 ## <a href="#variables-in-devcontainerjson" name="variables-in-devcontainerjson" class="anchor"> Variables in devcontainer.json </a>
 
 Variables can be referenced in certain string values in `devcontainer.json` in the following format: **${variableName}**. The following is a list of available variables you can use.
@@ -155,6 +169,7 @@ Variables can be referenced in certain string values in `devcontainer.json` in t
 | `${containerWorkspaceFolder}` | Any | The path that the workspaces files can be found in the container. |
 | `${localWorkspaceFolderBasename}` | Any | Name of the local folder that was opened in the `devcontainer.json` supporting service / tool (that contains `.devcontainer/devcontainer.json`). |
 | `${containerWorkspaceFolderBasename}` | Any | Name of the folder where the workspace files can be found in the container. |
+| `${devcontainerId}` | Any | Identifier derived from a set of container labels that uniquely idenity the dev container on a Docker host. It allows Features to refer to an identifier that is unique to the dev container they are installed into and that is stable across rebuilds.<br> The properties supporting it in devcontainer.json are: `name`, `runArgs`, `initializeCommand`, `onCreateCommand`, `updateContentCommand`, `postCreateCommand`, `postStartCommand`, `postAttachCommand`, `workspaceFolder`, `workspaceMount`, `mounts`, `containerEnv`, `remoteEnv`, `containerUser`, `remoteUser`, and `customizations`. |
 {: .table .table-bordered .table-responsive}
 
 ## Schema
diff --git a/_implementors/spec.md b/_implementors/spec.md
@@ -270,6 +270,25 @@ To resume the environment from a stopped state:
 
 Like during the create process, remote [environment variables](#environment-variables) and [user](#users) configuration should be applied to all created processes in the container (inclusive of `userEnvProbe`).
 
+## <a href="#parallel-exec" name="parallel-exec" class="anchor"> Parallel lifecycle script execution </a>
+
+Dev containers support a single command for each of its lifecycle scripts. While serial execution of multiple commands can be achieved with `;`, `&&`, etc., parallel execution deserves first-class support.
+
+All lifecycle scripts have been extended to support `object` types. The key of the `object` will be a unique name for the command and the value will be the `string` or `array` command. Each command must exit successfully for the stage to be considered successful.
+
+Each entry in the `object` will be run in parallel during that lifecycle step.
+
+### Example
+
+```json
+{
+  "postCreateCommand": {
+    "server": "npm start",
+    "db": ["mysql", "-u", "root", "-p", "my database"]
+  }
+}
+```
+
 # <a href="#definitions" name="definitions" class="anchor"> Definitions </a>
 #### <a href="#project-workspace-folder" name="project-workspace-folder" class="anchor"> Project Workspace Folder </a>
 
