Immediately disconnected after connect

[AndWass]

Using Windows and the shadow sample I immediately get disconnected after connecting and performing the TLS handshake. Wireshark shows the following output:

61	2.313533	192.168.15.167	54.154.234.121  TCP	      66        61059 → 8883 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 WS=256 SACK_PERM=1
69	2.358743	54.154.234.121	192.168.15.167	TCP	      66	8883 → 61059 [SYN, ACK] Seq=0 Ack=1 Win=26883 Len=0 MSS=1380 SACK_PERM=1 WS=256
70	2.358819	192.168.15.167	54.154.234.121	TCP	      54	61059 → 8883 [ACK] Seq=1 Ack=1 Win=66048 Len=0
72	2.373734	192.168.15.167	54.154.234.121	TLSv1.2	      257   	Client Hello
75	2.418502	54.154.234.121	192.168.15.167	TCP	      60    	8883 → 61059 [ACK] Seq=1 Ack=204 Win=28160 Len=0
76	2.420811	54.154.234.121	192.168.15.167	TCP	      1434	[TCP segment of a reassembled PDU]
77	2.420849	54.154.234.121	192.168.15.167	TLSv1.2	      1008      Server Hello, Certificate, Server Key Exchange, Certificate Request, Server Hello Done
78	2.420872	192.168.15.167	54.154.234.121	TCP	      54    	61059 → 8883 [ACK] Seq=204 Ack=2335 Win=66048 Len=0
79	2.427508	192.168.15.167	54.154.234.121	TLSv1.2	      1315 	Certificate, Client Key Exchange, Certificate Verify, Change Cipher Spec, Hello Request, Hello Request
83	2.474559	54.154.234.121	192.168.15.167	TLSv1.2	      105  	Change Cipher Spec, Hello Request, Hello Request
85	2.514520	192.168.15.167	54.154.234.121	TCP	      54        61059 → 8883 [FIN, ACK] Seq=1465 Ack=2386 Win=66048 Len=0
89	2.559828	54.154.234.121	192.168.15.167	TCP	      60        8883 → 61059 [FIN, ACK] Seq=2386 Ack=1466 Win=30720 Len=0
90	2.559864	192.168.15.167	54.154.234.121	TCP	      54        61059 → 8883 [ACK] Seq=1466 Ack=2387 Win=66048 Len=0

As you can see directly after line 83, a FIN ACK is sent closing the connection. Using OpenSSL and the following commandline openssl s_client -connect <endpoint>:8883 -CApath C:/AWS -cert <thingname>/cert.pem -key <thingname>/privkey.pem I instead get a working connection. I can also use for instance MQTT.fx to connect and communicate.

18012	638.716527	192.168.15.167	52.50.121.37	TCP	66	61408 → 8883 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 WS=256 SACK_PERM=1
18015	638.761403	52.50.121.37	192.168.15.167	TCP	66	8883 → 61408 [SYN, ACK] Seq=0 Ack=1 Win=26883 Len=0 MSS=1380 SACK_PERM=1 WS=256
18016	638.761490	192.168.15.167	52.50.121.37	TCP	54	61408 → 8883 [ACK] Seq=1 Ack=1 Win=66048 Len=0
18017	638.761721	192.168.15.167	52.50.121.37	TLSv1.2	362	Client Hello
18018	638.806599	52.50.121.37	192.168.15.167	TCP	60	8883 → 61408 [ACK] Seq=1 Ack=309 Win=28160 Len=0
18019	638.811071	52.50.121.37	192.168.15.167	TCP	1434	[TCP segment of a reassembled PDU]
18020	638.811142	52.50.121.37	192.168.15.167	TCP	1434	[TCP segment of a reassembled PDU]
18021	638.811182	192.168.15.167	52.50.121.37	TCP	54	61408 → 8883 [ACK] Seq=309 Ack=2761 Win=66048 Len=0
18022	638.811432	52.50.121.37	192.168.15.167	TLSv1.2	408	Server Hello, Certificate, Server Key Exchange, Certificate Request, Server Hello Done
18023	638.815252	192.168.15.167	52.50.121.37	TLSv1.2	1325	Certificate, Client Key Exchange, Certificate Verify, Change Cipher Spec, Encrypted Handshake Message
18032	638.862213	52.50.121.37	192.168.15.167	TLSv1.2	105	Change Cipher Spec, Hello Request, Hello Request
18033	638.904352	192.168.15.167	52.50.121.37	TCP	54	61408 → 8883 [ACK] Seq=1580 Ack=3166 Win=65792 Len=0

[JonathanHenson]

I suspect the server is hanging up because of an auth issue with your policy. Any chance you could show us your policy and a code snippet?

[JonathanHenson]

Also, could you possibly give it a try on linux or OSx for a sanity check to rule out a bug on our end with securechannel?

[JonathanHenson]

Oh, also, you’re using TLS 1.2, and I suspect, ALPN? But you’re using 8883 instead of 443? Since ALPN is supported, can you try using ALPN as our sample shows, but via 443?

[AndWass]

@JonathanHenson Thanks for swift reply! I don't have access to the policy I'm afraid; I don't have access to the endpoint itself since we are using a service provider to provide the actual backend. However as I said it works fine with MQTT.fx using TLS so that shouldn't be an issue though.

I have tried to build the package on Linux but my setup (I only want to build this as a static library since that is what we will use in the end) is getting build errors related to not finding libcrypto (even though OpenSSL and libcrypto is installed) which I am trying to work around.

Using the raw example (with no modifications at all) I get "Connection failed with unknown error code", so I had some modifications done. I get pretty much the same behaviour though; close immediately after a "change cipher spec, hello request, hello request"

[AndWass]

Got something working on Linux now but I get no output at all, neither connection rejects or anything.

[JonathanHenson]

On linux you need the header files for libcrypto, it’s usually in the ssl-dev package. Note: we don’t actually use OpenSSL, just libcrypto.

[JonathanHenson]

Assuming MQTT.fx is A java package, it does matter. For one we use native sdk providers, the java stuff isn’t using securechannel which is where I want to make sure we don’t have a bug. For two, java isn’t going to be using alpn by default.

Our samples use ALPN, and you’ll get very different behavior depending on the port you hit if we include the extension. So some sample code to repro would be really helpful.

Also, we’ve added some bug fixes to the crt package but haven’t update the docs for this repo yet (still testing) you may want to grab the latest tag and try that.

[JonathanHenson]

Just go ahead and assume that every implementation you find uses a different TLS implementation. For us, we use secure channel on windows, secure transport on Apple, and s2n everywhere else. Java has its own implementation, DotNet is similar to us other than using OpenSSL on the Unix distros. Python uses OpenSSL everywhere (even on windows).

So when tracking this stuff down, it working on one sdk doesn’t mean it’s not a TLS issue.

[graebm]

I think I encountered this same issue the other day.
I was using Windows, with TLS, and I was not setting an ALPN protocol list.
The code worked on Linux & Apple, but our Windows TLS implementation expected ALPN protocols and returned an error if they weren't set.

[AndWass]

Still haven't gotten anything to work. Using the following environments:

Windows 10 and MSVC 2017
OpenSUSE Leap 15 with g++ 7

In neither ones I get the shadow sample to show any output at all, its as if they don't do anything, I am using the shadow sample unmodified with latest master of both CRT and AWS sdk.

[JonathanHenson]

I just ran the samples again, and they're working for me. I strongly suspect this is due to the endpoints you're proxying too on your backend? Are you hitting the AWS IOT service anywhere? Is this a proxy or is it your own mqtt backend? The older java stuff is going to use a different set of extensions that are not compatible with the new endpoints, and this library is using modern extensions not compatible with the old endpoints. I'd love to help here, some additional information would speed up our ability to provide answers.

[AndWass]

Yes that might be related as well. I don't have access to the IoT service at all, all I know is that it is an AWS IoT service, and I have been given 2 address (one for ATS, one without). It might be that the endpoint is too old to work with this library then?

[JonathanHenson]

It’s not that. The problem is without more details I can’t confirm my suspicion. But if you’re hitting port 8883 but using ALPN you’re going to get hung up on. If you use 443 but don’t use ALPN, you’re going to get hung up on. If you use different endpoints without the CA, your negotiation will fail. If your policy doesn’t have connect permissions, the service will hang up on you immediately after you make the first call etc.... I need more details.

…

Sent from my iPhone

On Mar 1, 2019, at 2:18 AM, Andreas W ***@***.***> wrote: Yes that might be related as well. I don't have access to the IoT service at all, all I know is that it is an AWS IoT service, and I have been given 2 address (one for ATS, one without). It might be that the endpoint is too old to work with this library then? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.