-
Notifications
You must be signed in to change notification settings - Fork 325
Features/definitions collection proposal. #15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,114 @@ | ||
# Proposal | ||
|
||
The proposal is to arrive to an schema where different communities and organizations can create "collections" of features/definitions that they manage, support and present to users/ | ||
|
||
This collection data should have enough information so that different clients can search them and present them to users. | ||
|
||
For this we propose the existence of two different files: | ||
- Collection file: (devcontainer-collection.json) Groups of features/definitions that will be handled as the same origin (repository/publisher). Contains the metadata pertinent for each one, checksum and other information. | ||
edgonmsft marked this conversation as resolved.
Show resolved
Hide resolved
|
||
- Feature/Definition: (devcontainer-feature.json/devcontainer-definition.json) Information on a particular feature/definition required to add it to a collection. This file is optional and exists to support scenarios where one repository represents a collection and others represent each feature. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we'll want to make a decision here, either:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The way I envisioned it is actually both. Each feature/definition will be a release with a tar file with the source and json file with its description. An action stitches them together into a collection file. This should simplify the work that clients have to do to find and search collections while still allowing users to control where each feature/definition is actually stored and generated. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Got it. For now i've placed all the definitions into the a single file in vscode-dev-container, but could easily stitch that together with the existing GitHub Action https://github.com/microsoft/vscode-dev-containers/blob/main/devcontainer-collection.json |
||
|
||
This files would ideally be created automatically by a process that automatically adds information like: checksums, signatures, download links, etc. | ||
|
||
Clients would be able to select different sources for collections while also giving the option to users to add their own sources. | ||
|
||
## Collection file (devcontainer-collection.json) | ||
|
||
- Created as a release of a repository that either contains or represents groups of definitions/features from the same user/owner/publisher. (e.g. same organization in Github.) | ||
- Contains metadata for each definition/feature, repository origin, download link, checksum or signing information. | ||
- All referenced definitions/features would belong to the same user/organization. (e.g. download links would be the same organization in GitHub.) | ||
- References exact releases of each definition/feature. | ||
|
||
| Property | Type | Description | | ||
| :--- | :--- | :--- | | ||
| publisher | string | The organization/user that publishes this group of definitions/features. Must match with the repository user in github.| | ||
| repository | string | The repository where this collection is contained.| | ||
| version | string | Version information of this collection list.| | ||
| features | array | The list of features that are contained in this collection. Same information as the metadata file for the feature.| | ||
| definitions | array | The list of definitions contained in this collection. Same information as the metadata file for the definition.| | ||
|
||
Creating the collections file would make some changes over the metadata of the features/definitions where they are generated. | ||
|
||
- Ids would get the collection repository added to them. (e.g. for 'https://github.com/community/features' then 'community/features' to 'myfeature1' for a feature with that id) | ||
- The links to the release files would be added. | ||
- Date. | ||
|
||
The same way creating the definitions/feature file would also be part of the release process for it so that the following metadata can be added: | ||
- Version. | ||
- Checksum. | ||
- Signatures. | ||
- Date. | ||
|
||
## Definitions/features (devcontainer-feature.json/devcontainer-definition.json) | ||
|
||
### Common Properties | ||
|
||
| Property | Type | Description | | ||
| :--- | :--- | :--- | | ||
| id | string | Id of the feature/definition. The ids should be unique in the contest of the repository where they exist. | | ||
| displayName | string | Name of the feature/definition | | ||
| shortDescription | string | Optional one sentence description | | ||
| description | string | Description of the feature/definition | | ||
| metadata | any | Freeform data added by users for their own purposes. | | ||
| repositoryPath | string | Path to the repository and folder that contains the root of the code for the feature/definition. | | ||
| keywords | array | List of keywords relevant to a user that would search for this definition/feature. | | ||
| version | string | Versioning information for the feature/definition. | | ||
| files | array | List of links for the download files with checksums and optional signature information. | | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Individual files? (Not tar packages?) Will there be a tar per feature or a tar per collection? |
||
|
||
Chuxel marked this conversation as resolved.
Show resolved
Hide resolved
|
||
### Definitions | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we still want to call this 'definitions' :) |
||
|
||
| Property | Type | Description | | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. At a later point we could add an icon (that can be shown in the UI next to the name) and an optional link to a sample. |
||
| :--- | :--- | :--- | | ||
| type | string | singleContainer/orchestrated | | ||
| options | array | List of user selected options at the moment the user selects them in a client application. | | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Like the ones we currently parse out of the Dockerfile? |
||
| variants | array | Variations of the generated definition that contain different information. (e.g. with example code or data.) | | ||
|
||
Variants: | ||
| Property | Type | Description | | ||
| :--- | :--- | :--- | | ||
| name | string | Name of the variant | | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is each variant a definition by itself? (I guess that would require a link/pointer to that definition?) |
||
| description | string | Short description of the variant | | ||
|
||
### Features | ||
|
||
TODO: Add properties related to references/dependencies. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can this link to the other spec? |
||
|
||
# IDs | ||
|
||
TODO | ||
|
||
# Search | ||
|
||
Search would be implemented by client applications but the definitions themselves should include enough information for this to happen. Besides the typed properties defined above keywords could be: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Perhaps it is worth calling out that some keywords may be defined by clients as having an affect on UX or any "scoring" done for suggestions. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added that we should create a list of well-known keywords including those defined by clients. |
||
|
||
- Purpose. WebApp, WebService, IDE extension, etc. | ||
- Platform. Node, Ruby on Rails, .Net, etc. | ||
- Language. TS, JS, C#, Java, etc. | ||
- Other Apps. Databases, simulators, etc. | ||
- Dependencies. Bare metal, Azure, AWS, etc. | ||
- Organization. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Additional data that we might want to represent as separate properties (not as keywords) because we are using them programmatically (currently parsed from the definition's README.md):
|
||
|
||
This keywords would allow users to add search information easily like versions or more details to refine searches. (e.g. Python2 vs Python3). | ||
|
||
A list of well known keywords could be maintained, including suggested keywords and keywords defined by clients to add the search. | ||
|
||
# Example | ||
|
||
- User creates a group of features/definitions for his organization. | ||
- They are contained under multiple repositories under the "example" organization. | ||
_ The user adds an action to create release of all the different features/definitions. (A reference implementation should be included with this specification. This action would take a base metadata file, create the release tar file, and create the end metadata file with the download links, checksums, etc.) | ||
- The action could run automatically with every commit and could run any tests that are defined for it. | ||
- Optionally the action could create a PR on the collection repository to update the version. | ||
- User creates a repository under the same "example" organization to define the collection of features/ definitions. | ||
- This repository contains folders with files that point to the repositories that contain the features/definitions. (One folder for features, another for definitions.) | ||
- The user adds an action (Reference implementation.) that reads those folders and grabs the metadata for each feature/definition it finds in those repositories, consolidates it into 1 file and creates a release with it. | ||
- The action could be triggered by any changes to the folders that contain the features. (e.g. from the resulting merge of the PR created before.) | ||
|
||
# Remarks | ||
|
||
- By maintaining the rule of the user origin of a collection and files we can expose that information to the user reliably. | ||
- If the releases themselves are signed and the signature is valid we could also signal that information to the user. | ||
- The contents of all JSON files (metadata and collections) are considered unsafe and should be validated rigorously by any clients. | ||
- Trust is centered around the origin organization for a collection. | ||
- Security is handled by the user policies (e.g. who can merge or push) of the containing repository. | ||
- The community could define their own policies. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Am I missing the reference the concept of centralized 'index(es)'.
RIght now I have this flat file with a single reference to a single collection: https://github.com/joshspicer/devcontainer-index
We would then add pointers here to collections we "endorse". Others in the community would be able to produce their own indexes, which point whatever set of collections they would like. Additional indexes could be added to Codespaces via org policy, or via vscode by a setting.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see the indexes as more of a client feature, where clients define how to store and search it. Also, how users would add a reference to a collection to search their internal features for example.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is that where we would have the 3 collections from Codespaces, VS Code and the community to start with?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yea, I want us to publish an index file that will point to our three collections. All we need to do is hardcode a single index file, and that will point to the
N
collections we want to pull from by default.