Custom remote connections #304
Conversation
|
I like that idea. |
|
This is spot on with what we discussed several months ago, providing an abstracted interface to allow implementation of different transport protocols Ultimately I would like to see remoting into its own subsystem, and provide an interface to replace PSRP XML messages with some other protocol (gRPC, JSON-RPC, whatever) that just require you to implement functions like connect, send, receive, error, etc., so that PowerShell can be remoted to more easily from other languages (typescript, python, etc.) without having to implement and parse all of PSRP's heavyweight XML messages, however this is a good first step. |
There was a problem hiding this comment.
The overall scope seems nice but I do have some concerns how this custom connection will work with WSMan based connections. It's definitely the outlier in terms of how it works compared to the standard OutOfProc transports but to support a 3rd party OMI replacement it would have to be covered.
| ### Creating a connection | ||
|
|
||
| The PowerShell module implementing the new remote connection should supply all necessary cmdlets for creating and managing the connection. | ||
| In particular it should have a cmdlet that returns a `PSSession` object. |
There was a problem hiding this comment.
Yes. I feel many of the implementation details are trivial and don't need to be mentioned in this proposal.
There was a problem hiding this comment.
Just for the sake of the discussion, +1 on making the PSSession constructor public. My ideal solution would involve just building the PSSession object with lots of things exposed that we can override at will. Once the object is created, PowerShell just uses it like any other object, but under the hood it handles a custom transport.
| For example, both WSMan and SSH have services that do this. | ||
| Endpoints such as these are outside the scope of this feature proposal, and it is assumed that any custom connection scheme will include some sort of endpoint support. | ||
|
|
||
| PowerShell currently supports a 'server' running mode, where remoting protocol messages are channeled through the process stdIn/stdOut data streams. |
There was a problem hiding this comment.
Should the named pipe server also be mentioned, i.e. not the -s option but the PSHost. named pipe that pwsh creates for stuff like Enter-PSHostProcess/-CustomPipeName?
There was a problem hiding this comment.
I don't see any need to include other server modes in this proposal. To keep things simple, I just included proc/stdIO mode which I feel most implementers would use. Another endpoint option is to write a PowerShell WinRM plugin host since, since that is actually the only other way to create a PowerShell remoting endpoint instance. But it is very complex and doesn't provide much value in my opinion.
There was a problem hiding this comment.
Just my two cents: the way the OpenSSH client and server integration is currently done (subprocess + standard input/output streams) fits my needs perfectly except for the fact that there is no simple way to override the executable to launch (and its parameters). If this is exposed in some generic way I'd be happy, but I'm also happy if it's just exposed in the SSH remoting transport. I like it because it requires no modification to PowerShell whatsoever and no special classes, registration, etc.
There was a problem hiding this comment.
While that would be nice, it's not much more of a step removed to enable a class implementation and just ask the user to install a module with your custom transport. It's already been sadly discussed to death that modifying that existing provider isn't going to fly, however this solution allows it to be reimplemented to do what you want at least.
| ```powershell | ||
| # Create PowerShell endpoint session (with default shell configuration) | ||
| # Protocol handles messages through the child process stdIn/stdOut process pipes | ||
| $proc = Start-Process -FilePath pwsh.exe -ArgumentList '-servermode -nologo -noprofile' -PassThru |
There was a problem hiding this comment.
This is a bit of a misnomer, you cannot on Windows (AFAIK) hook into a processes stdio after it is created. You need to specify the pipes at creation to a handle of your choice. Start-Process can do this to an extent but only to a file rather than pipes that you control.
This doesn't rule out the named pipe communication I mentioned above though, just the stdio communication mentioned here.
There was a problem hiding this comment.
This is just to illustrate what command switches are used to run PowerShell in server mode. It is not intended to be an implementation. If we move forward, I intend to provide a full example of implementing a custom connection. But this is just a proposal document.
| // Override and implement this method that creates the custom client session transport with | ||
| // bound live connection data streams. | ||
| // This method is called from within the internal PowerShell remoting implementation. | ||
| public override BaseClientSessionTransportManager CreateClientSessionTransportManager( |
There was a problem hiding this comment.
I don't know how far in scope the RFC should be but it might be good to document what each of these options are for.
There was a problem hiding this comment.
I don't think it is needed in this proposal, but if we move forward all of this will be documented.
|
|
||
| // Start connection protocol. | ||
| // This is called by internal PowerShell remoting implementation. | ||
| public override void CreateAsync() |
There was a problem hiding this comment.
This is fine for OutOfProc based transports where the communication happens over 2 pipes or a single bidirectional pipe but WSMan is a bit of an outlier. It has a few special messages like the Create, Receive, Send, Command, Signal, Connect, etc WSMan messages. Is the expectation of that transport to decode the OutOfProc fragment that PowerShell will write to the stream like <Signal ../> and convert it to the relevant WSMan one? Are things like DisconnectAsync and ConnectAsync expected to be in scope for the transport manager?
There was a problem hiding this comment.
Yes, this proposal for custom connections is based on a fairly simple implementation of the PSRP, and so it relies on built-in implementations for command/signal events, and has no support for connect/disconnect or redirection.
I should probably add a section that mentions some of this, but I wanted to keep this proposal fairly short and general.
But you are right, in that this may be too simple for a reasonable WinRM client implementation. Such an implementation would have to essentially demux session, command, signal messages.
I didn't want to expose command transport abstractions or mediator functions as that greatly increases complexity and makes it harder to understand and implement. My aim is to make this as simple as possible while still being useable. But unfortunately the WinRM transport implementation is very complex.
There was a problem hiding this comment.
That's what I feared but it definitely understandable. I've tried to reconcile the client end of a transport between WSMan and OutOfProc based clients and settled on something like this (written in Python) https://github.com/jborean93/pypsrp/blob/c6667a44a34a3d64c89e80ccae646f25c537b25a/psrp/connection_info.py#L138-L311. Still that has some differences with how the transport manager in C# is set out but it's just some food for thought.
While it might be difficult to integrate a 3rd party WSMan client using the proposed model (in/out streams) it may still be doable. I can potentially see the custom connection reading the streams itself and converting it to the WSMan envelopes required. From a brief investigation it can easily send the WSMan create body based on the first Data packet and craft the WSMan Command/Signal/Delete bodies based on the Command, Signal, Close packets respectively.
The downside is the lack of support for disconnected operations. While that's not working on Linux today it would have been nice for it to be implemented at some point in the future. Even just opening up that possibility would be nice but maybe it's enough for an MVP and disconnected operations could be exposed at a future point if desired.
There was a problem hiding this comment.
Connection disconnect/connect can still be added at some point. But it is pretty complex (what do you do with streaming data?) and we would probably want a pretty good reason to do so.
There was a problem hiding this comment.
what do you do with streaming data?
I would say it's the same with WSMan, there is no more streaming between the client and server until something reconnects and there are server side policies to determine what to do with the accumulated data during that disconnected phase. It would require the server end to support this as well which as least with WSMan is already there. Other transports obviously would require extra work (if at all possible).
|
One other thing that would be great to see is supporting custom transports directly with the builtin PSSessison cmdlets like Just spitballing the idea but I could see it implemented by adding a parameter set that accepts
For example the RFC has the following example PS /Users/Paul> $sessions = New-MyCustomConnection -Computer $computerList -Credential $creds -UseMFA
PS /Users/Paul> $results = Invoke-Command -Session $sessions -File ./Scripts/WeeklyManagementTasks.ps1
PS /Users/Paul> Invoke-AnalyzeResults $results
PS /Users/Paul> $sessions | Remove-PSSessionIf the PS /Users/Paul> $connections = 1..4 | ForEach-Object { [MyCustomConnectionInfo]$_ }
PS /Users/Paul> $results = Invoke-Command -Connection $connections -File ./Scripts/WeeklyManagementTasks.ps1
PS /Users/Paul> Invoke-AnalyzeResults $results |
|
I like the idea of the RunspaceConnectionInfo parameter set. Not as intuitive as a specialized cmdlet that has parameters for the specific connection/authentication, but better than having a bunch of open remote connections, which is definitely resource heavy on the client. Another idea is to add support to PSSession object that allows dynamically managed session lifetime. So Invoke-Command can (via -ThrottleLimit parameter) can open the session and then close it after command completion. |
|
Yea I see them as complimentary scenarios each with their own advantages and disadvantages. Having both allows you to have a custom cmdlet that either returns a |
Me too. Original idea was to split monolith PowerShell Engine on subsystems so that we could remove a code from distribution if it is unneeded. Current proposal add more complicity and more confuse end users than add a value. As end user I'd prefer a transparent remoting. Why would end users think about underlying remoting transport and learn new cmdlets for every new remoting? We have an issue to add ComputerName parameter as common parameter to all cmdlets to implement remoting support to all cmdlets. In the case we could do |
That's mostly why I proposed the $connInfo = New-WSManConnectionInfo -ConnectionUri '...' -OtherParameters
# or
$connInfo = [WSManConnectionInfo]@{ConnectionUri = '...'; ...}
Invoke-Command -Connection $connInfo, $otherConn, $etc { echo 'hi' }Like with
The trouble with it accepting a string is how would it determine whether to use WSMan, SSH, insert custom transport, based on just the name itself. You would have to have some way to encapsulate the provider as well as allow options the caller wishes to set for that provider into a string. You could maybe accept a hashtable but discoverability somewhat becomes harder in this scenario compared to a cmdlet that can give this "connection info". Therein lies what I think the problem with accepting a simple string, it works great where things are just simple but as soon as you start to stray from this path it's a lot more complicated. |
If we want innovations we should start from UX considerations:
Only then should we talk about the details of implementation. Ideally users shouldn't think about what transport is used and scripts should works on any transport without script change. |
|
I just don’t know how you could have a connection without the user knowing how they want to connect. How should PowerShell determine that you want to use WSMan/SSH/custom? The mere fact that something is custom would indicate the custom implementation deals with something PowerShell knows nothing about. I don’t think we can eliminate the caller choosing how to connect as the caller is the one who knows the context behind it. For the other points you’ve mentioned I don’t see them as insurmountable. Like with any modules if someone has installed it then IMO it’a a reasonable assumption that they also know about how it can generate the connection/session object for you. |
I agree with Jordan on this one - there is no way to make custom transports fully transparent to users to the point where they do not have to explicitly specify how they want to connect. We can try and make it easier to specify, but we can't get rid of everything - we need at least a transport protocol name somewhere, like "ssh", "wsman" or "my-custom-protocol" + protocol-specific parameters. I think that if we try and make it automagic for the user, we could easily derail this entire discussion into not making it do what it should do: expose ways to override the PowerShell remoting transport with a different, custom transport. |
| [WindowsMachine1]: PS C:\> Users\UserName> | ||
| ``` | ||
|
|
||
| This example installs the `PSWSManConnection` module from the PowerShell Gallery onto a macOS machine. |
There was a problem hiding this comment.
Just to clarify: in this example, the PowerShell module publishing and installation is not mandatory, since the module provides New-PSWSManPSSession functions that returns a PSSession object that PowerShell can use. I am personally for an approach that would focus solely on making it possible to construct a PSSession object usable by PowerShell - how that PSSession object is constructed can be complex, but wrapped into a simple function like New-PSWSManSession. In many cases it's possible that the implementation of this function could fit in a single PowerShell file, and it might be possible to just copy/paste it into a new PowerShell terminal without installing anything.
There was a problem hiding this comment.
Yes, that is the goal of this RFC which is to enable community code to construct a PSSession object that can be used by PowerShell
| The PowerShell engine provides WinRM specific connection objects, `WSManConnectionInfo`, to create a connection through WinRM. | ||
| It also contains a transport implementation for transferring protocol messages and data through the WinRM connection. | ||
|
|
||
| #### Example - SSH remoting |
There was a problem hiding this comment.
I'd like to add that SSH remoting is by far the most "decoupled" PowerShell remoting transport in the sense that it it does everything with standard input/output streams on both sides - there is really nothing much that is directly tied to OpenSSH in particular beyond the fact that it expects to call ssh.exe with specific command-line arguments. This is the transport type that has a lot of potential for generalization and easy integration with unmanaged languages like C/Rust using an external executable that behaves like OpenSSH.
There was a problem hiding this comment.
Eh, most of the OutOfProc ones are like this. SSH just uses the stdio ports of the process to communicate but the named pipe stuff is a bidirectional pipe. The proposal here is to allow people to implement their own connections and the mechanism for exchanging data is using 2 .NET streams leading to the same thing really.
The only real outlier here is WSMan, the rest of the connection all revolve around that OutOfProc methodology with a read and write mechanism to exchange data.
In Windows PowerShell I do There are many examples of how to implement transparent extensions based on negotiations. See RDP. Or SMTP https://cr.yp.to/smtp/ehlo.html |
|
@iSazonov I think this is reasonable, but how would the "registration" work? Would we require the user to import the module ahead of time, or would there be some kind of special Register-Transport command that would add it to a config somewhere? How do we handle the order of transports (e.g. typically I use my custom gRPC transport to connect to the stated computername server but this one time I want to use WSMAN instead)? And what if my transport for instance requires a certificate to authenticate and I need to specify which one? Should I supply cmdlets to configure it? What if the user needs to make a runtime exception? Remoting doesn't currently do this today with e.g. SSH transport vs WBEM transport, it always uses the WBEM transport and doesn't try to "detect" SSH if it is available as a transport, you explicitly have to call it out by using |
I think what you're really asking for is a way to make custom transports usable for PowerShell implicit remoting, where you only specify the target computer name and it automatically handles everything for you. There are a number of challenges to solve that may not be appropriate for all custom transports, the first one being that you need to detect which custom transport to use, and the second is that credentials need to be implicit as well. While this is relatively easy with Integrated Windows Authentication in a Windows domain, it's not very doable for most other transport types. As for RDP-style negotiation, this requires... a negotiation protocol. The goal of custom transports is not replace the transport entirely, not to provide a new protocol on top of which we should tunnel a custom protocol. RDP can tunnel virtual channels protocols that the core protocol doesn't know much about beyond a virtual channel name, but that virtual channel exchange is still done over RDP. That won't fly if we want to swap the entire transport layer by something else. |
I do have a suggestion that could make implicit remoting a lot more ergonomic: accept a protocol scheme in the target hostname, like ssh://computer, wsman://computer, wayk://computer. This would still require credentials to be available in the current context for implicit remoting, but it would be a relatively simple way to select a remoting protocol. As for registration, we may not even truly need it: the protocol could become part of an expected function name like this: ssh:// -> New-SshPSSession If we accept all valid protocol schemes, we should support characters like +/-/. but it can still be mapped automatically to a valid function name: jet+tcp:// -> New-JetTcpPSSession I'm just using a bunch of custom protocol schemes from my own products for reference here, but you get the idea. All you'd need is to declare a New-PSSession function in the current context and it could get picked up automatically based on the URL scheme of the target. |
Just to clarify, I think you mean this: |
@iSazonov that is exactly like Having some sort of fallback mechanism where PowerShell will try multiple connection is just a bit too magical for my liking. Just like with hostnames and what you want to connect to I firmly believe how you want to connect is a decision solely in the domain of the caller and not PowerShell should try and determine for you.
I was going to suggest a similar thing like |
|
@jborean93 I think the simplest is usually best - I think we should accept the protocol scheme, but also properly parse the entire parameter as a valid URI. This includes an optional port field, and even query parameters. For 99% of use cases where credentials are present in the current context, the only thing you will ever need is a correctly-constructed URL to indicate where to connect and how. As for passing credentials, they obviously shouldn't be passed in the URL. The first solution would be to have them available in the current context (like SSH keys, or Integrated Windows Authentication) for cases where it is possible. Dynamic credential prompting could be done by a New-CustomPSSession function, but should ideally be avoided as it would only be compatible with interactive scenarios, and incompatible with implicit remoting. Another option would be to pass an opaque hashmap of parameters to the New-CustomPSSession constructor, but then we need to add a new parameter in New-PSSession, and find a way to pass this new parameter in implicit remoting scenarios. This is where explicit credentials could be passed. |
|
IMO if you willing to write a string with the protocol in there why not just call the cmdlet directly that gives you the object anyway. Take completion with No need to add messy string parsing logic that may or may not work with across all future custom transport types that complicates already complicated code. No need to force module authors to provide some sort of registration implementation during module load complicating an already complicated process for what I argue is only really nice in the demoware scenarios. |
|
I'm with @jborean93, and I think you should be able to have some sort of Set-PSTransportOption command that will let you specify the order of transports to attempt, otherwise as transports are loaded as modules they get inserted into the top of the order of methods to try, and a transport should have some way to gracefully report that it is either not running or failed to connect before the next transport method is attempted, ideally something that is suppressable because noone wants to see a red wall of text every time they try an implicit command unless all connection methods fail (at which point the exceptions for all could be rolled into an innerexception for TransportConnectivityFailed or whatever) A URL scheme has bad discoverability compared to something like a PSSessionTransportOption object specifically for Transports, however allowing a URL scheme to be passed as a string as a shortcut that just parses it into a PSSessionTransportOption object would be fine I think, to help allow for those one-off adjustments you may need to do like specify a different port or username. For configuration/preauth/etc. of transports, I think it should be left up to transport authors to implement similar to SecretManagement, but ideally some sort of .sshconfig style configuration where you can have per-host overrides would become a de-facto standard if not a prescribed standard. |
A list of transports to attempt in order can very easily cause extremely poor UX when the first one on the list doesn't work. How long should it try connecting using transport X before giving up? 30 seconds? 10 seconds? 1 second? I would advocate for an approach where you end up with a single protocol to try, not a list. |
|
@awakecoding Yeah I'm aware of that but that's what the set-pstransportoption would be for, if you want to force a particular protocol all the time. By default for this RFP no transport modules will be loaded so it will always still just be WSMAN, and you basically have to explicitly import a module to use that transport, so you are by essence specifying that anyways. Timeout would be left up to the transport author to configure but the encouragement should require a pretty aggressive default, allowing it to be relaxed if necessary for higher latency connections. |
|
Guys, I wonder why you are discussing implementation details. The starting point is the UX and the experience we already have with the implementation of an alternative SSH remoting. The main drawback of the current implementation is that we have to change scripts and applications to make them work on a different transport. That's why I say that the ideal UX is to be transparent - easy to create, easy to deploy, easy to configure, easy to use. As for implementation. We live last 5 years without "portable" remoting (I mean WSMan is not supported on Unix, SSH is not on all Windows - bad UX). So de-facto it is not too critical :-). And so we could start experimenting in next (non-LTS) version and try to decouple remoting to a subsystem. This will give us an understanding of how we could build in new transports. |
|
@iSazonov I don't understand why we shouldn't be discussing possible implementation details - this, after all, is a request for comments, and we definitely need to go beyond "we want custom PowerShell remoting transports" without discussing the how at least on the surface. We all have specific pain points we want to make sure are going to be considered in the design. While I consider PowerShell implicit remoting to be secondary for my use cases, it appears to be your primary point of concern. I think we can find a fair common ground that allows explicit custom transport methods to be used implicitly with minimal changes to scripts. If you want it to be transparent, then we have no choice but to discuss how it could be implemented transparently. As a side note, I am happy with the direction the discussions are taking. There seems to be consensus on creating a PSSession object that PowerShell can consume without knowing the custom transport implementation underneath. As for the rest, it is really discussion on how to connect the dots on top of that and make it all ergonomic for every day usage and our combined use cases (explicit vs implicit, etc). |
Because the devil is in the details :). I agree the UX is where it starts, and having implicit remoting "just work" by just specifying a computername most of the time and have Powershell "figure it out" by attempting custom transports in order is the ideal solution, and in most cases the defaults will be fine, but in the case where the defaults are not fine (you need to specify a port/certificate/whatever to connect), how that is addressed is still part of the UX and thus why it is being discussed here. |
|
Changes to PowerShell remoting user experience is outside the scope of this RFC. But it is something that can certainly be proposed in a separate RFC. This RFC is limited to proposing a way to support other types of remoting connections via an external module. I am happy to discuss implementation details when they relate to the feasibility or usefulness of the proposal. |
|
|
||
| #### Example - Local process remoting | ||
|
|
||
| PowerShell jobs (`Start-Job`) use PowerShell remoting but the remote connection is to a local child process instead of a remote machine. |
There was a problem hiding this comment.
Is this done via named pipes or stdout/stdin? I would love to see how a named pipe session could be created.
There was a problem hiding this comment.
If you are talking about today you can do this already with https://docs.microsoft.com/en-us/dotnet/api/system.management.automation.runspaces.namedpipeconnectioninfo?view=powershellsdk-7.0.0.
$proc = Start-Process pwsh -PassThru
$proc.Id
# Since v6 this can be an actual name of a named pipe, otherwise pwsh calculates the
# expected name from the PID.
$connInfo = [System.Management.Automation.Runspaces.NamedPipeConnectionInfo]::new($proc.Id)
$rs = [RunspaceFactory]::CreateRunspace($connInfo)
$rs.Open()
$ps = [PowerShell]::Create()
$ps.Runspace = $rs
$ps.AddScript('$pid').Invoke()
$rs.Dispose()I even have a testing script I use which uses named pipes to log the PSRP data exchanged which uses named pipe as communication https://github.com/jborean93/pypsrp/blob/asyncio/New-PSSessionLogger.ps1.
The trouble here is that there is no public way to create a PSSession with this mechanism, You can use reflection to create the PSSession object or just stick with the .NET API but that rules out using things like Invoke-Command. I'm hoping that this RFC makes it easier to 1 utilise named pipes in your own custom transports and 2 make it easy to use them with cmdlets that accept a PSSession object.
There was a problem hiding this comment.
Start-Job uses stdIn/stdOut. However, there is a named pipe connection/transport implementation as well that you can look at.
…C00XX-Custom-Remote-Connections.md
…tom-Remote-Connections.md
|
@PowerShell/powershell-committee reviewed this and we are accepting this for an experimental feature in 7.3 |
No description provided.