No connection is available to service this operation

[naruto1227]

No connection is available to service this operation: SETEX string:testkey; IOCP: (Busy=1,Free=999,Min=2,Max=1000), WORKER: (Busy=0,Free=32767,Min=2,Max=32767), Local-CPU: n/a

[naruto1227]

There is no load on the service

[naruto1227]

2.0.519

[dceravigupta]

@zhanghaiboshiwo are you using a Redis cache cluster like Azure Redis?

[naruto1227]

No，It is simple

[naruto1227]

only one

[gadeweever]

@zhanghaiboshiwo might be related to #1374 but not quite sure.

Can you see if this reproes in 2.1.0? It was released yesterday.

[tebeco]

I just stumbled upon this when trying to run AspNetCore tests for Microsoft.AspNetCore.SignalR.StackExchangeRedis
Issue still present with 2.1.0

issue : dotnet/aspnetcore#20040
PR to udpate to 2.1.0 ongoing : dotnet/aspnetcore#20042

[NickCraver]

@tebeco Are you saying it's reproduce-able in some config in 2.1.0? If so, I'm very, very curious!

[tebeco]

the repro for me is
Make sure Docker4Win runs (i'm on 2.2.0.4 + wsl2 backend enabled)
PR : dotnet/aspnetcore#20040

git clone https://github.com/tebeco/AspNetCore/tree/update-servicestack-redis
cd AspNetCore
restore.cmd
build.cmd
cd src/SignalR
startvs.cmd

Ctrl+Shift+B to trigger all test detection
then go to Test Explorer > find and run RedisEndToEnd
you should see 15 Test KO

[tebeco]

here is what i see in the
(Same info different readability)

Test Logs

[3/21/2020 12:14:19.304 AM] [xUnit.net 00:00:16.68]     Microsoft.AspNetCore.SignalR.StackExchangeRedis.Tests.RedisEndToEndTests.CanSendAndReceiveUserMessagesFromMultipleConnectionsWithSameUser(transportType: LongPolling, protocolName: "json") [FAIL]
[3/21/2020 12:14:19.305 AM] [xUnit.net 00:00:16.68]       Microsoft.AspNetCore.SignalR.HubException : An unexpected error occurred invoking 'EchoUser' on the server. RedisConnectionException: No connection is available to service this operation: PUBLISH Microsoft.AspNetCore.SignalR.StackExchangeRedis.Tests.EchoHub:user:userA; UnableToConnect on 172.17.0.2:6379/Interactive, Initializing/NotStarted, last: NONE, origin: BeginConnectAsync, outstanding: 0, last-read: 5s ago, last-write: 5s ago, keep-alive: 60s, state: Connecting, mgr: 10 of 10 available, last-heartbeat: never, global: 5s ago, v: 2.1.0.1; IOCP: (Busy=0,Free=1000,Min=16,Max=1000), WORKER: (Busy=5,Free=32762,Min=16,Max=32767), Local-CPU: n/a
[3/21/2020 12:14:19.306 AM] [xUnit.net 00:00:16.68]       Stack Trace:
[3/21/2020 12:14:19.309 AM] [xUnit.net 00:00:16.68]         C:\dev\github\tebeco\aspnetcore\src\SignalR\clients\csharp\Client.Core\src\HubConnection.cs(776,0): at Microsoft.AspNetCore.SignalR.Client.HubConnection.InvokeCoreAsyncCore(String methodName, Type returnType, Object[] args, CancellationToken cancellationToken)
[3/21/2020 12:14:19.309 AM] [xUnit.net 00:00:16.68]         C:\dev\github\tebeco\aspnetcore\src\SignalR\common\Shared\ForceAsyncAwaiter.cs(64,0): at System.Threading.Tasks.ForceAsyncAwaiter`1.GetResult()
[3/21/2020 12:14:19.310 AM] [xUnit.net 00:00:16.68]         C:\dev\github\tebeco\aspnetcore\src\SignalR\clients\csharp\Client.Core\src\HubConnection.cs(390,0): at Microsoft.AspNetCore.SignalR.Client.HubConnection.InvokeCoreAsync(String methodName, Type returnType, Object[] args, CancellationToken cancellationToken)
[3/21/2020 12:14:19.310 AM] [xUnit.net 00:00:16.68]         C:\dev\github\tebeco\aspnetcore\src\Testing\src\TaskExtensions.cs(53,0): at Microsoft.AspNetCore.Testing.TaskExtensions.TimeoutAfter(Task task, TimeSpan timeout, String filePath, Int32 lineNumber)
[3/21/2020 12:14:19.311 AM] [xUnit.net 00:00:16.68]         C:\dev\github\tebeco\aspnetcore\src\SignalR\server\StackExchangeRedis\test\RedisEndToEnd.cs(111,0): at Microsoft.AspNetCore.SignalR.StackExchangeRedis.Tests.RedisEndToEndTests.CanSendAndReceiveUserMessagesFromMultipleConnectionsWithSameUser(HttpTransportType transportType, String protocolName)
[3/21/2020 12:14:19.311 AM] [xUnit.net 00:00:16.68]         --- End of stack trace from previous location ---

and Test Summary

Microsoft.AspNetCore.SignalR.StackExchangeRedis.Tests.RedisEndToEndTests.CanSendAndReceiveUserMessagesFromMultipleConnectionsWithSameUser(transportType: LongPolling, protocolName: "json")
   Source: RedisEndToEnd.cs line 95
   Duration: 15.1 sec

  Message: 
    Microsoft.AspNetCore.SignalR.HubException : An unexpected error occurred invoking 'EchoUser' on the server. RedisConnectionException: No connection is available to service this operation: PUBLISH Microsoft.AspNetCore.SignalR.StackExchangeRedis.Tests.EchoHub:user:userA; UnableToConnect on 172.17.0.2:6379/Interactive, Initializing/NotStarted, last: NONE, origin: BeginConnectAsync, outstanding: 0, last-read: 5s ago, last-write: 5s ago, keep-alive: 60s, state: Connecting, mgr: 10 of 10 available, last-heartbeat: never, global: 5s ago, v: 2.1.0.1; IOCP: (Busy=0,Free=1000,Min=16,Max=1000), WORKER: (Busy=5,Free=32762,Min=16,Max=32767), Local-CPU: n/a
  Stack Trace: 
    HubConnection.InvokeCoreAsyncCore(String methodName, Type returnType, Object[] args, CancellationToken cancellationToken) line 776
    ForceAsyncAwaiter`1.GetResult() line 64
    HubConnection.InvokeCoreAsync(String methodName, Type returnType, Object[] args, CancellationToken cancellationToken) line 390
    TaskExtensions.TimeoutAfter(Task task, TimeSpan timeout, String filePath, Int32 lineNumber) line 53
    RedisEndToEndTests.CanSendAndReceiveUserMessagesFromMultipleConnectionsWithSameUser(HttpTransportType transportType, String protocolName) line 111
    --- End of stack trace from previous location ---

[tebeco]

i do confirm that it does reproduce and VS it telling me that 2.1.0 is loaded

[NickCraver]

@tebeco This didn't click back then but was an underlying issue resolved in #1947. Look out for the v2.5 release to have "make sure subscriber is connected before Connect(Async) comes back", "heartbeat keeps pub/sub connection alive properly", "better subscription restore handling on reconnect", and "by-default backlog to handle disconnects more gracefully". In total, this should resolve a lot of problems - see the #1947 PR for more details!