Enable PKCS11 Support

[NoMoreFood]

• Enable use of PKCS11 library files by adjusting master configuration file.
• Modified dlsym() to return a void pointer instead of an int which is consistent with POSIX. The previous return type caused an issue with 32-bit builds with PKCS11 enabled.

[cloudhan]

Any chance will this PR be merged? I need PKCS#11 support to ssh into the server which only support smartcard authentication.

[manojampalam]

@cloudhan can you enlighten me on how your scenario works? Do you have a PKCS provider on Windows? What did you mean by "server only support smartcard authentication" ?

[cloudhan]

@manojampalam My company provide me with an USBKey (aka, the smartcard, similar to Yubikey) to login to the dev server. To my understanding, it basically is ssh public key auth. I can retrieve a public key from the USBKey and register it to the server. To login, I need the private key, but there is no way to retrieve the private key literal (by design). So, I either use the MS-CAPI or PKCS#11 for login authentication . However, OpenSSH ship with Windows 10 does not support. Currently I'm using the PuTTY-CAC suggested by OpenSC. But its font rendering is pretty awful.

If this pr can be merged, it can slightly improve my workflow in the future. (as well as a much border audience, as you can see from the short list of OpenSC)

[NoMoreFood]

@manojampalam The way that PKCS support works in OpenSSH is that you pass in a vendor-provided DLL (or .SO file on Linux) that enables the capability to interface with the hardware token. The exported functions in the DLL must match the PKCS11 specification in order to be used by OpenSSH. PuTTY-CAC (of which I am the maintainer) is an add-on to PuTTY to provide this functionality by either using the same PKCS DLL or by using the CAPI subsystem on Windows (which uses the Windows minidriver for the token). Admittedly, I do not control the font rendering part of the code --- that's part of the upstream PuTTY code.

[NoMoreFood]

@manojampalam Thoughts on moving forward with this?

[jcotton42]

This is also something I'd like to see

[NoMoreFood]

@bingbing8 I adjusted my commit for simplicity by defining the PKCS11 compiler flag in the central configuration file and it looks like the appveyor tests are giving me "The specified module 'pester' was not loaded because no valid module file was found in any module directory." I doubt it's related to my change. Any idea what might be going on here?

[bingbing8]

@NoMoreFood , if you check the set up log, pester is not installed successfully. https://ci.appveyor.com/api/buildjobs/u0fl8r630g54pf00/artifacts/TestSetupLog.txt

[NoMoreFood]

@bingbing8 Understood. Shouldn't that be something that the test setup does automatically or is there something additional I should be doing in my commits to install pester?

[bingbing8]

@NoMoreFood , the tests download and install it. The failure could due to temporary chocolatey server issue, network issue. or the particular version does not exist any more. Previously we install pester 3.4.6 due to a pester bug at that time. Please refer https://github.com/PowerShell/openssh-portable/blob/latestw_all/contrib/win32/openssh/OpenSSHTestHelper.psm1#L366. looks like the version still exists https://chocolatey.org/packages/pester/3.4.6.
you can either push a changes, or close/reopen the PR to trigger another test run if it is temporary chocolatey server or network issue.

[bingbing8]

@NoMoreFood , I trigger the build again by close/reopen the PR, all checks passed.

[mattmccull]

@manojampalam Do we have a estimate as to when this will be merged? This is something I'm interested in.

[spencercw]

This PR exposes an existing issue with the dlerror() implementation which causes a crash if the PKCS#11 DLL cannot be loaded. See #379 for the fix.

[spencercw]

Does this need to be stdcall? It would probably make more sense to leave this with the default calling convention like the other functions in this file.

[NoMoreFood]

You're right. We do not. Commit adjusted.

[jmfrank63]

Is this pull request going to be accepted? Would be a great addition.

[keliansb]

Please accept this PR, I'd like this to be implemented too.