Skip to content

Secure tunnel v3 update samples #569

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 52 commits into from
Apr 14, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
eec7341
linux-integration-tests.yml
sbSteveK Apr 7, 2023
51f9eb4
yml update
sbSteveK Apr 7, 2023
ff6e2bc
run setup-linux.sh
sbSteveK Apr 7, 2023
d780a0f
install cmake
sbSteveK Apr 7, 2023
270d94c
notes
sbSteveK Apr 7, 2023
b30a7bc
test
sbSteveK Apr 7, 2023
ab108c9
test
sbSteveK Apr 7, 2023
67a0018
integration-tests.sh
sbSteveK Apr 7, 2023
508a734
chmod integration-tests.sh
sbSteveK Apr 10, 2023
7f34428
check what's set
sbSteveK Apr 10, 2023
66241be
secure tunneling tests directory
sbSteveK Apr 10, 2023
138e262
print first
sbSteveK Apr 10, 2023
686f20f
open secure tunnel
sbSteveK Apr 10, 2023
6552a2a
open and store tunnel tokens
sbSteveK Apr 10, 2023
cdedf37
work on integration test
sbSteveK Apr 10, 2023
5ef64ad
update
sbSteveK Apr 10, 2023
598e1d1
wip
sbSteveK Apr 11, 2023
40a17a3
remove sleep
sbSteveK Apr 11, 2023
4345125
testing
sbSteveK Apr 11, 2023
73d76cf
test
sbSteveK Apr 11, 2023
c63a9b5
test
sbSteveK Apr 11, 2023
f699b7c
setup fix
sbSteveK Apr 11, 2023
3eb3ed4
test fix
sbSteveK Apr 11, 2023
926c99f
sleep test
sbSteveK Apr 11, 2023
46ac3f6
fix sleep
sbSteveK Apr 11, 2023
ad9e097
separate destination secure tunnel
sbSteveK Apr 11, 2023
6c0a651
start/stop
sbSteveK Apr 11, 2023
294f465
fix
sbSteveK Apr 11, 2023
7f3075b
two clients
sbSteveK Apr 11, 2023
4366853
fix
sbSteveK Apr 11, 2023
f604a47
more testing
sbSteveK Apr 11, 2023
7162fb8
logging
sbSteveK Apr 11, 2023
a6a96fd
segfault check
sbSteveK Apr 11, 2023
96aa7cc
updates
sbSteveK Apr 12, 2023
584eecf
fixed segfault
sbSteveK Apr 12, 2023
31873c5
message sending checks
sbSteveK Apr 12, 2023
d043cdd
comments
sbSteveK Apr 12, 2023
4686bb0
edits to secure tunnel docs
sbSteveK Apr 12, 2023
e5ba5a9
clean up service id storage on error
sbSteveK Apr 12, 2023
2d4904a
Merge branch 'secure-tunnel-v3-tests' into secure-tunnel-v3-documenta…
sbSteveK Apr 12, 2023
1a51947
Merge branch 'secure-tunnel-v3' into secure-tunnel-v3-documentation
sbSteveK Apr 13, 2023
d3ca6fc
document fixes
sbSteveK Apr 13, 2023
66e073d
update samples and readme
sbSteveK Apr 14, 2023
17fb4b1
readme update
sbSteveK Apr 14, 2023
e9501d9
size_t instead of int
sbSteveK Apr 14, 2023
7ceb6e9
sleep functionality
sbSteveK Apr 14, 2023
d805918
revert generated files
sbSteveK Apr 14, 2023
16969ca
fix warning
sbSteveK Apr 14, 2023
464b716
warnings as errors
sbSteveK Apr 14, 2023
313ba6b
with -> With for builder functions
sbSteveK Apr 14, 2023
5a7f638
bold some README lines
sbSteveK Apr 14, 2023
c38da78
with -> With
sbSteveK Apr 14, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 18 additions & 4 deletions documents/Secure_Tunnel_Userguide.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ When a WebSocket upgrade request fails to connect, this callback will return an
### OnConnectionShutdown
When the WebSocket connection shuts down, this callback will be invoked.

### OnSendDataComplete
## OnSendMessageComplete
When a message has been completely written to the socket, this callback will be invoked.

### OnMessageReceived
Expand All @@ -55,6 +55,12 @@ When a stream is started by a Source connected to the Destination, the Destinati
### OnStreamStopped
When an open stream is closed, this callback will be invoked and return the stopped stream's information.

### OnConnectionStarted
When a connection start message is received and a new active connection is established, the Destination will invoke this callback and return the connection information.

### OnConnectionReset
When a connection has ended either in error or closed intentionally by the secure tunnel peer, the client will invoke this callback and return the connection information.

### OnSessionReset
When the Secure Tunnel service requests the Secure Tunnel client fully reset, this callback is invoked.

Expand Down Expand Up @@ -143,13 +149,14 @@ if(!secureTunnel->Stop()){
fprintf(stdout, "Failed to stop the Secure Tunnel connection session. Exiting..\n");
}
```
# Multiplexing
You can use multiple data streams per Secure Tunnel by using the [Multiplexing](https://docs.aws.amazon.com/iot/latest/developerguide/multiplexing.html) feature.
# Multiplexing and Simultaneous TCP Connections
You can use multiple data streams per Secure Tunnel by using the [Multiplexing and Simultaneous TCP Connections](https://docs.aws.amazon.com/iot/latest/developerguide/multiplexing.html) features.


## Opening a Secure Tunnel with Multiplexing
To use Multiplexing, a Secure Tunnel must be created with one to three "services". A Secure Tunnel can be opened through the AWS IoT console [Secure Tunnel Hub](https://console.aws.amazon.com/iot/home#/tunnelhub) or by using the [OpenTunnel API](https://docs.aws.amazon.com/iot/latest/apireference/API_Operations_AWS_IoT_Secure_Tunneling.html). Both of these methods allow you to add services with whichever names suit your needs.
## Services Within the Secure Tunnel Client
On a successfull connection to a Secure Tunnel, the Secure Tunnel Client will invoke the `OnConnectionSuccess` callback. This callback will return `ConnectionSuccessEventData` that will contain any available Service Ids that can be used for multiplexing. Below is an example of how to set the callback using the Secure Tunnel Builder and check whether a Service Id is available.
On a successfull connection to a Secure Tunnel, the Secure Tunnel Client will invoke the `OnConnectionSuccess` callback. This callback will return `ConnectionSuccessEventData` which contains any available Service Ids that can be used for multiplexing. Below is an example of how to set the callback using the Secure Tunnel Builder and check whether a Service Id is available.
```cpp
// Create Secure Tunnel Builder
SecureTunnelBuilder builder = SecureTunnelBuilder(...);
Expand Down Expand Up @@ -191,6 +198,8 @@ builder.WithOnConnectionSuccess([&](SecureTunnel *secureTunnel, const Connection
```
## Using Service Ids
Service Ids can be added to outbound Messages as shown below in the Send Message example. If the Service Id is both available on the current Secure Tunnel and there is an open stream with a Source device on that Service Id, the message will be sent. If the Service Id does not exist on the current Secure Tunnel or there is no currently active stream available on that Service Id, the Message will not be sent and a Warning will be logged. The `OnStreamStarted` callback is invoked when a stream is started and it returns a `StreamStartedEventData` which can be parsed to determine if a stream was started using a Service Id for Multiplexing. Incoming messages can also be parsed to determine if a Service Id has been set as shown above in the [Setting Secure Tunnel Callbacks](#setting-secure-tunnel-callbacks) code example.
## Using Connection Ids
Connection Ids can be added to outbound Messages as shown below in the Send Message example. If there is an active stream currently open using the combination of the Service Id and Connection Id, the message will be sent. If a Connection Id is not set on an outbound message, a Connecion Id of 1 is assumed and applied to the Message. When additional streams are activated, the `OnConnectionStarted` callback is invoked and returns a `ConnectionStartedEventData` which can be parsed to determine the Connection Id of the newly activated stream. A Connection Id will also be present in the `StreamStartedEventData` that is returned when the `OnStreamStarted` callback is invoked.
# Secure Tunnel Operations

## Send Message
Expand All @@ -201,11 +210,16 @@ Crt::String serviceId_string = "ssh";
Crt::String message_string = "any payload";

ByteCursor serviceId = ByteCursorFromString(serviceId_string);
uint32_t connectionId = 1
ByteCursor payload = ByteCursorFromString(message_string);

// Create Message
std::shared_ptr<Message> message = std::make_shared<Message>();
// Add a Service Id
message->withServiceId(serviceId);
// Add a Connection Id
message->withConnectionId(connectionId);
// Add a payload
message->withPayload(payload);

// Send Message
Expand Down
65 changes: 13 additions & 52 deletions samples/secure_tunneling/secure_tunnel/README.md
Original file line number Diff line number Diff line change
@@ -1,72 +1,33 @@
## Secure Tunnel
# Secure Tunnel

[**Return to main sample list**](../../README.md)

This sample uses AWS IoT [Secure Tunneling](https://docs.aws.amazon.com/iot/latest/developerguide/secure-tunneling.html) Service to connect a destination and a source with each other through the AWS Secure Tunnel endpoint using access tokens using the [V2WebSocketProtocol](https://github.com/aws-samples/aws-iot-securetunneling-localproxy/blob/main/V2WebSocketProtocolGuide.md). For more information, see the [Secure Tunnel Userguide](../../../documents/Secure_Tunnel_Userguide.md)

Your IoT Core Thing's [Policy](https://docs.aws.amazon.com/iot/latest/developerguide/iot-policies.html) must provide privileges for this sample to connect, subscribe, publish, and receive. Below is a sample policy that can be used on your IoT Core Thing that will allow this sample to run as intended.

<details>
<summary>(see sample policy)</summary>
<pre>
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iot:Publish",
"iot:Receive"
],
"Resource": [
"arn:aws:iot:<b>region</b>:<b>account</b>:topic/$aws/things/<thing_name>/tunnels/notify"
]
},
{
"Effect": "Allow",
"Action": [
"iot:Subscribe"
],
"Resource": [
"arn:aws:iot:<b>region</b>:<b>account</b>:topicfilter/$aws/things/<thing_name>/tunnels/notify"
]
},
{
"Effect": "Allow",
"Action": [
"iot:Connect"
],
"Resource": [
"arn:aws:iot:<b>region</b>:<b>account</b>:client/test-*"
]
}
]
}
</pre>

Replace with the following with the data from your AWS account:
* `<region>`: The AWS IoT Core region where you created your AWS IoT Core thing you wish to use with this sample. For example `us-east-1`.
* `<account>`: Your AWS IoT Core account ID. This is the set of numbers in the top right next to your AWS account name when using the AWS IoT Core website.
* `<thingname>`: The name of your AWS IoT Core thing you want the device connection to be associated with

Note that in a real application, you may want to avoid the use of wildcards in your ClientID or use them selectively. Please follow best practices when working with AWS on production applications using the SDK. Also, for the purposes of this sample, please make sure your policy allows a client ID of `test-*` to connect or use `--client_id <client ID here>` to send the client ID your policy supports.
This sample uses AWS IoT [Secure Tunneling](https://docs.aws.amazon.com/iot/latest/developerguide/secure-tunneling.html) Service to connect a destination or a source Secure Tunnel Client to an AWS Secure Tunnel endpoint using access tokens using the [V3WebSocketProtocol](https://github.com/aws-samples/aws-iot-securetunneling-localproxy/blob/main/V3WebSocketProtocolGuide.md). For more information, see the [Secure Tunnel Userguide](../../../documents/Secure_Tunnel_Userguide.md)

## How to run

Create a new secure tunnel in the AWS IoT console (https://console.aws.amazon.com/iot/) (AWS IoT/Manage/Tunnels/Create tunnel) and retrieve the destination and source access tokens. (https://docs.aws.amazon.com/iot/latest/developerguide/secure-tunneling-tutorial-open-tunnel.html). Once you have these tokens, you are ready to open a secure tunnel.

### Destination Mode

To run the sample with a destination access token in destination mode (default), you can use the following command:

``` sh
./secure_tunnel --region <region> --access_token_file <path to destination access token>
./secure_tunnel --signing_region <signing_region> --access_token_file <path to destination access token>
```

However, for this sample to work, it will also need another instance of the sample running in source mode. You can run another instance of the sample in source mode using the same command, but adding the `--localProxyModeSource` flag:
The sample will create a Secure Tunnel connection and remain connected in `DESTINATION MODE` and will echo any messages it receives through the Secure Tunnel back to the Source Device.

### Source Mode

While the focus of the Secure Tunnel Client for the IoT Device SDK is to connect with Secure Tunnels in `DESTINATION MODE` we also support connecting in `SOURCE MODE`. The token file should be the Source Token in this instance and you must add the `--localProxyModeSource` flag:

``` sh
./secure_tunnel --region <region> --access_token_file <path to source access token> --localProxyModeSource
./secure_tunnel --signing_region <signing_region> --access_token_file <path to source access token> --localProxyModeSource
```

Then two samples will then connect to each other through the AWS Secure Tunnel endpoint and establish a stream through which data can be transmitted in either direction.
The sample will create a Secure Tunnel connection in `SOURCE MODE` and will open a stream using an available `Service Id`. It will then send n messages on the opened stream. It will then create a new simultaneous TCP connection on the stream and send an additional n messages on the new TCP connection. It will then exit.

### Proxy
Note that a proxy server may be used via the `--proxy_host` and `--proxy_port` argument. If the proxy server requires a user name and password to connect, you can use `--proxy_user_name` and `--proxy_password` to in the sample to pass the required data to the sample.
8 changes: 4 additions & 4 deletions samples/secure_tunneling/secure_tunnel/main.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -223,13 +223,13 @@ int main(int argc, char *argv[])
/* Echo message on same service id received message arrived on */
if (message->getServiceId().has_value())
{
echoMessage->withServiceId(message->getServiceId().value());
echoMessage->WithServiceId(message->getServiceId().value());
}

/* Echo message on the same connection id received message arrived on */
if (message->getConnectionId() > 0)
{
echoMessage->withConnectionId(message->getConnectionId());
echoMessage->WithConnectionId(message->getConnectionId());
}

secureTunnel->SendMessage(echoMessage);
Expand Down Expand Up @@ -368,10 +368,10 @@ int main(int argc, char *argv[])
/* If the secure tunnel has service ids, we will use one for our messages. */
if (m_serviceId.has_value())
{
message->withServiceId(m_serviceId.value());
message->WithServiceId(m_serviceId.value());
}

message->withConnectionId(connectionId);
message->WithConnectionId(connectionId);

secureTunnel->SendMessage(message);

Expand Down
19 changes: 11 additions & 8 deletions samples/secure_tunneling/tunnel_notification/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,13 @@

[**Return to main sample list**](../../README.md)

This sample uses the AWS IoT [Secure Tunneling](https://docs.aws.amazon.com/iot/latest/developerguide/secure-tunneling.html) Service to receive a tunnel notification.
This sample uses an MQTT Client and the AWS IoT [Secure Tunneling](https://docs.aws.amazon.com/iot/latest/developerguide/secure-tunneling.html) Service to receive a tunnel notification and then connects to the Secure Tunnel using a Secure Tunnel Client.

This sample requires you to create a tunnel for your thing. See [instructions here](https://docs.aws.amazon.com/iot/latest/developerguide/secure-tunneling-tutorial.html) on how to create a tunnel. You can also read more about secure tunneling in the [Secure Tunnel Userguide](../../../documents/Secure_Tunnel_Userguide.md).
This sample requires you to create a Secure Tunnel for your thing. See [instructions here](https://docs.aws.amazon.com/iot/latest/developerguide/secure-tunneling-tutorial.html) on how to create a tunnel. You can also read more about secure tunneling in the [Secure Tunnel Userguide](../../../documents/Secure_Tunnel_Userguide.md).

On startup, the sample will wait until it receives, and then displays the tunnel notification.
This sample requires a certificate and key file using Mutual TLS (mTLS). On startup, the device connects to the server using the certificate and key files and subscribes to a specific topic generated along with the Thing Name to listen for any Secure Tunnels that are opened for that specific Thing. When a Secure Tunnel is created for the Thing, it will receive a publish with the Secure Tunnel information which it can then use to create a Secure Tunnel Client and connect to the Secure Tunnel Service in `DESTINATION MODE`. AWS IoT will publish the notification upon creation of a new tunnel and will retransmit a new notification if instructed to "Generate new access tokens".

Your IoT Core Thing's [Policy](https://docs.aws.amazon.com/iot/latest/developerguide/iot-policies.html) must provide privileges for this sample to connect, subscribe, and receive. Below is a sample policy that can be used on your IoT Core Thing that will allow this sample to run as intended.

<details>
<summary>(see sample policy)</summary>
Expand All @@ -21,7 +23,7 @@ On startup, the sample will wait until it receives, and then displays the tunnel
"iot:Receive"
],
"Resource": [
"arn:aws:iot:<b>region</b>:<b>account</b>:topic/$aws/things/<thing_name>/tunnels/notify"
"arn:aws:iot:<b>region</b>:<b>account</b>:topic/$aws/things/<b>thing_name</b>/tunnels/notify"
]
},
{
Expand All @@ -30,7 +32,7 @@ On startup, the sample will wait until it receives, and then displays the tunnel
"iot:Subscribe"
],
"Resource": [
"arn:aws:iot:<b>region</b>:<b>account</b>:topicfilter/$aws/things/<thing_name>/tunnels/notify"
"arn:aws:iot:<b>region</b>:<b>account</b>:topic/$aws/things/<b>thing_name</b>/tunnels/notify"
]
},
{
Expand All @@ -53,10 +55,11 @@ Replace with the following with the data from your AWS account:

Note that in a real application, you may want to avoid the use of wildcards in your ClientID or use them selectively. Please follow best practices when working with AWS on production applications using the SDK. Also, for the purposes of this sample, please make sure your policy allows a client ID of `test-*` to connect or use `--client_id <client ID here>` to send the client ID your policy supports.

## How to run

Create a new secure tunnel in the AWS IoT console (https://console.aws.amazon.com/iot/) (AWS IoT/Manage/Tunnels/Create tunnel). Once you have the tunnel open, you can run the sample with the following command:
</details>

## How to run
To Run this sample, use the following command:
``` sh
./tunnel-notification --endpoint <endpoint> --cert <path to the certificate> --key <path to the private key> --thing_name <thing name>
```
Once the MQTT Client is connected, create a new secure tunnel in the AWS IoT console (https://console.aws.amazon.com/iot/) (AWS IoT/Manage/Tunnels/Create tunnel) for the Thing.
Loading