Unable to connect to Timestream from using endpoint (.Net)

[Anna-Ignatov]

Describe the bug

Trying to connect to Timestream from .Net using query endpoint - we are using private VPC

Expected Behavior

Connect to Timestream

Current Behavior

This is configuration we sent:
RegionEndpoint = RegionEndpoint.EUWest1,
Timeout = tsConfig.Timeout,
MaxErrorRetry = tsConfig.MaxErrorRetry,
EndpointDiscoveryEnabled=true
And received
Amazon.Runtime.AmazonClientException: Failed to discover the endpoint for the request. Requests will not succeed until an endpoint can be retrieved or an endpoint is manually specified
We tried to specify also
ServiceURL= "https://query-cell1.timestream.eu-west-1.amazonaws.com"
And received the same error.
We also tried to set EndpointDiscoveryEnabled= false - with the same result
In case we do no specify service URL we receive
The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing

Reproduction Steps

var awsConfig=new AmazonTimestreamQueryConfig
{
RegionEndpoint = RegionEndpoint.EUWest1,
Timeout = tsConfig.Timeout,
MaxErrorRetry = tsConfig.MaxErrorRetry,
EndpointDiscoveryEnabled=false
// ServiceURL= "https://query-cell1.timestream.eu-west-1.amazonaws.com"
};
Services.AddScopedAsImplementedInterfaces(awsConfig);

We have a log of configuration:
public AwsTimeSeriesRepository(TimeSeriesConfig config, IAmazonTimestreamQuery timestreamClient,
ILogger logger)
{
_timestreamQueryBuilder = new TimestreamQueryBuilder(config);
_timestreamClient = timestreamClient;
_logger = logger;
_config = config;
_logger.LogInformation("config={config}", JsonConvert.SerializeObject(_config));
_logger.LogInformation("AWSConfig={awsConfig}", JsonConvert.SerializeObject(timestreamClient.Config));
}

        Please, see the AWS configuration we received in log:

"ServiceId": "Timestream Query",
"RegionEndpointServiceName": "query.timestream",
"ServiceVersion": "2018-11-01",
"UserAgent": "aws-sdk-dotnet-coreclr/3.7.301.22 ua/2.0 os/linux#5.10.217.205 md/ARCH#X64 lang/.NET_Core#7.0.20 md/aws-sdk-dotnet-core#3.7.304.17 api/Timestream Query#3.7.301.22",
"Profile": null,
"AWSTokenProvider": {
"ProfileName": null
},
"SignatureMethod": 1,
"SignatureVersion": "4",
"UseAlternateUserAgentHeader": false,
"RegionEndpoint": null,
"ServiceURL": "
https://query-cell1.timestream.eu-west-1.amazonaws.com/"
,
"UseHttp": false,
"AuthenticationRegion": null,
"AuthenticationServiceName": "timestream",
"MaxErrorRetry": 1,
"IsMaxErrorRetrySet": true,
"LogResponse": false,
"ReadEntireResponse": false,
"BufferSize": 8192,
"ProgressUpdateInterval": 102400,
"ResignRetries": false,
"AllowAutoRedirect": true,
"LogMetrics": false,
"DisableLogging": false,
"DefaultConfigurationMode": 5,
"ProxyCredentials": null,
"Timeout": "1.00:00:00",
"UseDualstackEndpoint": false,
"UseFIPSEndpoint": false,
"IgnoreConfiguredEndpointUrls": false,
"DisableRequestCompression": false,
"RequestMinCompressionSizeBytes": 10240,
"ClientAppId": null,
"ThrottleRetries": true,
"CorrectedUtcNow": "2024-06-20T12:44:10.776782+00:00",
"ClockOffset": "00:00:00",
"DisableHostPrefixInjection": false,
"EndpointDiscoveryEnabled": false,
"EndpointDiscoveryCacheLimit": 1000,
"RetryMode": 0,
"FastFailRequests": false,
"CacheHttpClient": true,
"HttpClientCacheSize": 1,
"ReadWriteTimeout": null,
"EndpointProvider": {},
"ProxyHost": null,
"ProxyPort": 0,
"MaxConnectionsPerServer": null,
"HttpClientFactory": null
}

Possible Solution

No response

Additional Information/Context

No response

AWS .NET SDK and/or Package version used

AWSSDK.TimestreamQuery" Version="3.7.301.22"

Targeted .NET Platform

.Net7

Operating System and version

AmazonLinux

[ashishdhingra]

@Anna-Ignatov Good morning. Thanks for opening the issue. Service endpoints lists the service endpoints for TimeStream. Could you try or confirm below:

• Enable verbose logging using code below and share/examine the detailed log for exact error:

  Amazon.AWSConfigs.LoggingConfig.LogResponses = Amazon.ResponseLoggingOption.Always;
  Amazon.AWSConfigs.LoggingConfig.LogTo = Amazon.LoggingOptions.SystemDiagnostics;
  Amazon.AWSConfigs.AddTraceListener("Amazon", new System.Diagnostics.ConsoleTraceListener());

  You may develop a simple console application and run it on your EC2 instance in private VPC to isolate the configuration.
• Check your VPC for security group rules that might be blocking outgoing traffic.
• If you have VPC endpoint configured, then refer Configuring VPC endpoint service connectivity (documentation is for KMS service, but should work for Timestream service) to allow service connectivity.

Thanks,
Ashish

[amraninoam]

I am getting the same issue as the OP: Trying to connect to timestream with no internet connection over vpc endpoint.
I reckon the issue is that the .Net SDK allows configuration of the service url, which I found available only when there's an internet connection. The service url (i.e. https://query.timestream...) is different from endpoint url (i.e. https://query-cell1.timestream... ), which can be accessed internally, without internet connection.
Regardless of EndpointDiscoveryEnabled setting true or false, the SDK is calling the service url to find out the endpoint url.
Thus, resulting in inability to use the .net sdk in an environment without internet access.

@ashishdhingra, what is the expected way to workaround no internet connectivity?

[dscpinheiro]

@Anna-Ignatov and @amraninoam Did this exception start happening after you updated the AWSSDK.TimestreamQuery package? There was a change on how the SDK resolves endpoints in version 3.7.100, so I'm trying to understand if this was an unintended side effect.

[amraninoam]

@dscpinheiro , I have tried it with the latest SDK, you can find my code here, and the specific SDK versions here
I also tried with lower SDKs on a different branch , it result in the same behaviour as 3.7.300.