draft of winget yaml

[SteveL-MSFT]

Example of what a configuration doc may look like authored by end user to be used by an orchestrator

[SteveL-MSFT]

This should include versioning

[JohnMcPMS]

Consider some form of revision number being present. Imagine a scenario where there is some external manifest being used to drive one of the resources. If that file changes, but this one does not, it would be difficult to determine that anything has changed.

[michaeltlombardi]

Can you expand on this a bit? With this file representing the user-defined desired state, the merging of this definition with any other information would happen during the building of the interchange data blob - and at that point the handling service has the context it needs to report/act.

You could give this document a human-friendly revision/name but you'd still want to use something like a hash of the blob to in the service side to know whether anything changed more definitively, I think.

[JohnMcPMS]

I think this ties very much into the next comment about "author-time-known files", so I will largely answer there.

[JohnMcPMS]

Regarding variables: I think we need either an automatic variable similar to $PSScriptRoot or a defined semantic for what the . directory means within a file. Either way, that value should be "the directory containing the file" so that things like ($PSConfigDir)\..\file.txt can be used to reference a file that is known to be collocated.

[michaeltlombardi]

To clarify, you mean an automatic variable to get to the root for where this file lives? Can you explain this use-case a little bit?

My naive thought is for author-time-known files¹, if I want to move those to a specific location (or download them, or process them somehow for further use) I would use a specific resource for that.

Footnotes

1. Files that I'm keeping with my configuration definition rather than generated by a resource during the set action, such as adding a pre-defined config file for an app that I'm going to copy in place after install, or a remote resource I'm going to use. Their location is known at author time even if their content isn't (and presumably I trust the content). ↩

[JohnMcPMS]

Yes, that is what I mean about the variable.

I'm confused on your example as well, in that I don't see how you can reference such an author-time-known file from the configuration file unless there is some defined mechanism for stating where it should be found in relation.

For example, Visual Studio has .vsconfig files that contain the components needed by a solution. They are already a perfectly good mechanism for informing the user via the UI that new component dependencies have been added. If I similarly wanted to use this configuration mechanism to set up a build agent, I could reference said .vsconfig as the set of components necessary:

    - resource: VisualStudioComponents
      settings:
        productId: Microsoft.VisualStudio.Product.Enterprise
        channelId: VisualStudio.17.Release
        vsconfigFile: ($PSConfigDir)\..\.vsconfig

[gaelcolas]

I see two options here:

• something like facter which would build (retrieve and cache) some local data defined before running the config.
• having again a loose object defined with kind/spec, so that the value of vsconfigFile can be resolved in some ways, depepending of what the kind has implemented...

In case of the vsconfig file, this one would have to be dropped by earlier config, or assume it was there before by other means.

[JohnMcPMS]

It might be interesting to enable user-driven parameters as well. For instance, if the desired state is "any one of these options", then the parameter node could be something like:

- variable: UserSelection
  default: A
  options:
    - A
    - B
    - C

Things get complicated if the user wants "any of these is fine for Test, but if you call Set, use B". That feels like something to push on the resource to implement AnyOptionOkForTest: true if that kind of flexibility is acceptable.

[michaeltlombardi]

Can you elaborate on this a little? Are you envisioning using this key to define variables for a user to override at a higher layer or as a way to pass values to be reused resource definitions in the file, or something else?

[JohnMcPMS]

Are you envisioning using this key to define variables for a user to override at a higher layer

Yes, or even potentially to require a user selection (no default value) at the higher layer.

as a way to pass values to be reused resource definitions in the file

Yes, but I'm not sure what other use the parameters could have.

I get that overly parameterized desired state gets into a strange place, but sometimes the desired state just needs one item from a set of compatible alternatives to function. Allowing those alternatives to be carried with the DSC rather than defined in some documentation seems like a worthwhile improvement (and enables code to be written around offering the alternatives where it might otherwise require manual intervention).

[gaelcolas]

Having user-provided configuration data is interesting, but that means the two data must be separated and merged before use.
The merged version is what's used, but we need to have metadata to know where the data is coming from (which merged layer).
Think of it as Hiera having a hierarchy of the admin provided config data and the user provided config data, and user-provided data can only be explicitly allowed by admin, potentially with merge rules...
A can of worms, but that'd be great value.

[michaeltlombardi]

My instinct is that the user data/merging is a higher-order tool concern specific to the tool chain that takes you from "user-authored config in a text file" to this interchange format. That allows you to work with any given higher order tool instead of trying to get the CM tools to conform to a single data-merging model.

To your point that would allow winget to implement a data-merging model for their toolchain while still allowing someone to use datum or hiera in theirs.

[gaelcolas]

Not necessarily.
My point is that some data is managed at authoring time by the config author (there you might want some tool), some at execution time, asked to the user (i.e. parameters as in my example on my PR).

[michaeltlombardi]

Oh, you mean a way to mark a configuration setting/value as runtime-mungable in the interchange document so tools know whether (and contextually) if values can be prompted up to the runtime user?

[gaelcolas]

Yes, but add just one more thing, the data requested to the user ought to be cached...
So you have "user data" (populated when first run happens) and "assigned" configuration.
So for each subsequent runs, the user data is merged, without asking the user.
Higher level tools ought to collect and store this data for the user, so that a system could be restored to a user's preference.

[michaeltlombardi]

My instinct is that the caching behavior/etc should be the concern of the higher order tool, but I definitely see the value in marking an item as taking user input for the interchange document (so the higher order tools can do something with it).

[michaeltlombardi]

Potential later improvement, but not critical / sensibly deferred I think:

We could support the allOf (logical AND), anyOf (logical OR), and oneOf (logical XOR) keys defined for JSON schema composition. We could also possibly support the not (logical NOT) key from that definition as well, ensuring that we back off a run under a specific context.

My naive thought is that if assertions is an array it's implicitly allOf. If it's a map, the only valid keys are these compositional keys. This setting could also be recursive, so you could combine assertions with (basic) branching logic:

# Valid when foo is installed as v1 or foo is installed at v2+ and bar is v3+
# Invalid if foo is installed at v0 or not at all
# Invalid if foo is installed at v2+ and bar isn't installed or is v2 or lower
assertsions:
  anyOf:
    - resource: assert-package
      settings:
        name: foo
        version: "[1.0-2.0)"
    - allOf:
        - resource: assert-package
          setttings:
            name: foo
            version: "2.0"
        - resource: assert-package
          settings:
            name: bar
            version: "(3.0,)"

While I see a lot of useful flexibility here, it's also possible that this tempts people to solve a problem at the wrong layer (building targeting into the configuration document instead of having the targeting decided at the higher level). It might be best to preserve assertions only for things that the configuration author knows will cause runtime failures.

[JohnMcPMS]

We have also discussed this some in regards to my comment about parameters and compatible options. Using composition rather than a parameter simplifies the functionality (Test makes more sense), but the authoring/maintainability is somewhat painful.

I agree on your reservations and think that targeting should be handled at a higher layer (ex. separate files for separate scenarios), but it is definitely a spectrum.

[gaelcolas]

I disagree with that approach, strongly.
You're basically suspending the agent to converge to a defined state based on some pre-conditions (assertions above), aka not converging to its assigned state.

But you already have created the policy document (compiled), assigned it to the system, and now you're saying don't do it unless...
If you do so, you're not managing the state of your system... and you'll need 20 or more of those "policies", ending up removing the policy-driven benefit of configuration management.

And if you have 20 or so conditions to evaluate before you can start understanding in which state your system is, you've lost the benefit of the abstraction.

I do see the value of "WaitFor" with a software/version to match before continuing in the convergence, but the end goal should be to be in the desired state.

[michaeltlombardi]

@gaelcolas that goes directly into my concerns about this functionality (emphasis added here):

tempting people to solve a problem at the wrong layer (building targeting into the configuration document instead of having the targeting decided at the higher level). It might be best to preserve assertions only for things that the configuration author knows will cause runtime failures.

I still see strong value in a "fail early before changing system state" mechanism but I do see anti-patterns if this functionality is flexible enough to tempt people away from using a higher order tool/abstraction for targeting.

[Bpoe]

Since this is Powershell DSC, why not allow conditions to be expressed in Powershell instead creating another DSL?

"$assert-package.version -gt 3.0"

[gaelcolas]

@Bpoe: Because it's not just PowerShell DSC, afaik (and is in need for clarifications).
As I understand the goal is to have a tool agnostic "utility" that invokes resources, some being PowerShell DSC Resources, but potentially others...
See #8 and #7 where I try to say that it's confusing.

[JohnMcPMS]

Is the plan to be strict on directives? That does provide the possibility of the guarantee that "if you can provide it, it will be implemented". I think that if we want to be strict here, it might be good to provide a defined extensibility point for users to attach arbitrary data to a resource. That allows for some form of user defined processing to occur on data that we have explicitly stated is outside of the scope of DSC.

[gaelcolas]

In my experience, I quite like the way you can define pretty much anything with kind/spec representation...
I don't like the name "directive", but maybe there's reasons I don't know about in this name (first of all, naming is hard).
But if we call that bit "resource"

- instanceName: osversion
  resource:
    kind: pwsh_module
    spec:
      name: Microsoft.OSVersion # example of specifying a module name
      repository: PSGallery # example of specifying a repository name
      confirm: true # example where you might require Windows update, but need confirmation from user initiated by the orchestrator (do we need support for resources to prompt for confirmation?)
      signerThumbprint: AFBF0B8B6A18F7E23CCA1DDCD0AC1A55B4035173 # thumbprint of the signer cert valid for multiple versions

That's if we want something specific to be resolved for resource:, but if we know the "thing" we reference can be resolved by default, you ought to be able to addres simply like so:

- instanceName: osversion
  resource: Microsoft.OSVersion

[SteveL-MSFT]

@JohnMcPMS you mean like allowing the user to add notes for their own purpose not passed to a resource or orchestration? Wouldn't comments serve that purpose or you think they want something that's part of the object/yaml?

Discussed with @gaelcolas his feedback on this and it centers around being able to uniquely identify a resource when a system has multiple of the same name (imagine a scenario where an ITPro is defining config for IIS and another for SQL, but both want to use FileResource but different implementations). I think we agreed that this can happen, but should be optional to add filters to make the resource identify specific. Minimally, if orchestration detects a conflict, it should fail and report to the user the conflict. VSCode tooling can help identify the conflict and have a selector with filters. However, that only detects it on the authoring system and not the one being deployed onto unless the orchestration also wraps all the used resources as part of the configuration payload.

[michaeltlombardi]

I think the way most CM tools uniquely identify a resource is by combining the resource type (namespace/name) and key properties - for example, in Puppet, resources have a namevar that is either defined by a single key on an instance, the concatenated value of multiple keys on an instance, or a custom string built from the properties of the instance.

So in the case of a FileResource, if the key is the Path property, you couldn't define two instances of FileResource with Path set to C:/Foo/Bar.yaml.

I definitely see some value in ensuring that the conversion process adds that value to this document as instanceName where the rest of the information is needed to tell the consumer of the document which resource implementation it's supposed to use/look for.

Thinking about it from the perspective of tools that don't necessarily know about each other and only speak through the interchange format, the consumer of the interchange needs enough information to know if it has the resource available (potentially, to retrieve it) and to pass the specific resource actions/data to it.

[gaelcolas]

If the InstanceID is "calculated", we need instanceName so we (humans) can reference an instance in a DependsOn. I'm actually in favor of that, because having the namevar for the instance name makes it easier to relate a config to the script/DSL that produced it, while in a MOF the "generated" ResourceID is mixed with other info making it messy and not human readable...
In the DSC DSL, iirc, the instanceID is also used to uniquely reference resources that were nested in a Composite Resource.
ex:

instance of MSFT_FileDirectoryConfiguration as $MSFT_FileDirectoryConfiguration1ref
{
ResourceID = "[File]C\\Test\\Dev-Environment::[FilesAndFolders]FilesAndFolders";
 Type = "Directory";
 Ensure = "Present";
 DestinationPath = "C:\\Test\\Dev-Environment";
 ModuleName = "PSDesiredStateConfiguration";
 SourceInfo = "::6::2::File";

ModuleVersion = "1.0";

 ConfigurationName = "RootConfiguration";

};

[Bpoe]

Why not just let this be a user assigned name/id field? Ansible allows just a user defined named that uniquely identifies the block programmatically as well as a human friendly string. DevOps pipeline have a displayName field.

[gaelcolas]

That's what I suggest, just a string.
A human writing a config could give it a pet name...
A higher level tool would "generate" it based on some of its own rule (like the DSC DSL does today).

[JohnMcPMS]

Should this document include the specification for signing a configuration file? Presumably it would eventually, but I can understand not worrying about it now.

YAML doesn't have a canonical embedded signature format, although there are specifications by independent users. One interesting thing that YAML supports is stream boundaries, so one could actually embed the signature(s) as YAML in subsequent stream(s) within the same file.

[gaelcolas]

Added my 2p here...
I think we'd get more from a meeting at this point, but it could just be that I'm late in the game.

[gaelcolas]

This is a "Wait for" type, right?
So the "wait for software version at" resource has the option to not support a SET.
But that's not a specific behaviour for the "agent".

[michaeltlombardi]

This is a good point. I think in some cases it's being used as a less-intelligent (as in, has access to less complex decision making) targeting mechanism. "Don't do this if the node doesn't match prerequisites because the run will deterministically fail, even if someone applied this configuration to the node on purpose"

An example would be trying to apply a Fedora configuration to a Windows 11 machine or trying to apply a configuration that requires a specific version of the OS - fail early and obviously without changing system state.

I think that usage is separate from "wait until this condition (possibly set in this run) is true."

[gaelcolas]

I'm not completely against this feature (and left it in my PR), but it's moved to the agent to implement, not the resource graph to converge to.

I'd argue that if you try to apply a config to the wrong OS, it's an issue with your assignment.
How could you converge to a state that has little to do with you...

[michaeltlombardi]

My original intent with having this in the draft was a short-circuit functionality - a way to cut a run short if it gets called in a context you know at author-time can't possibly succeed, without ever changing system state.

[gaelcolas]

I disagree with that approach, strongly.
You're basically suspending the agent to converge to a defined state based on some pre-conditions (assertions above), aka not converging to its assigned state.

But you already have created the policy document (compiled), assigned it to the system, and now you're saying don't do it unless...
If you do so, you're not managing the state of your system... and you'll need 20 or more of those "policies", ending up removing the policy-driven benefit of configuration management.

And if you have 20 or so conditions to evaluate before you can start understanding in which state your system is, you've lost the benefit of the abstraction.

I do see the value of "WaitFor" with a software/version to match before continuing in the convergence, but the end goal should be to be in the desired state.

[gaelcolas]

Having user-provided configuration data is interesting, but that means the two data must be separated and merged before use.
The merged version is what's used, but we need to have metadata to know where the data is coming from (which merged layer).
Think of it as Hiera having a hierarchy of the admin provided config data and the user provided config data, and user-provided data can only be explicitly allowed by admin, potentially with merge rules...
A can of worms, but that'd be great value.

[gaelcolas]

Suggested change

	- resource: osversion
	- InstanceName: osversion

We're defining the name of the instance, whatever its type, if I understand correctly.
so if we can call this specific instance of Microsoft.OSVersion, osversion, it's easier to understand, no?

[gaelcolas]

In my experience, I quite like the way you can define pretty much anything with kind/spec representation...
I don't like the name "directive", but maybe there's reasons I don't know about in this name (first of all, naming is hard).
But if we call that bit "resource"

- instanceName: osversion
  resource:
    kind: pwsh_module
    spec:
      name: Microsoft.OSVersion # example of specifying a module name
      repository: PSGallery # example of specifying a repository name
      confirm: true # example where you might require Windows update, but need confirmation from user initiated by the orchestrator (do we need support for resources to prompt for confirmation?)
      signerThumbprint: AFBF0B8B6A18F7E23CCA1DDCD0AC1A55B4035173 # thumbprint of the signer cert valid for multiple versions

That's if we want something specific to be resolved for resource:, but if we know the "thing" we reference can be resolved by default, you ought to be able to addres simply like so:

- instanceName: osversion
  resource: Microsoft.OSVersion

[gaelcolas]

I see two options here:

• something like facter which would build (retrieve and cache) some local data defined before running the config.
• having again a loose object defined with kind/spec, so that the value of vsconfigFile can be resolved in some ways, depepending of what the kind has implemented...

In case of the vsconfig file, this one would have to be dropped by earlier config, or assume it was there before by other means.

[Bpoe]

properties seems like an extraneous and unneeded level of nesting to me. What are your thoughts on its purpose?

[SteveL-MSFT]

Closing this PR as it's out of date with what's going on with WinGet and also a move towards ARM syntax for the schema