Hanging query for Firestore

[thomasdao]

Operating System

Both Mac and Windows

Browser Version

Chrome, Electron Browser window

Firebase SDK Version

10.7.0, 10.7.1

Firebase SDK Product:

Firestore

Describe your project's tooling

Plain Electron app

Describe the problem

This is the new ticket for hanging query issue, follow up from #7771 and #7652

When update Firebase to 10.7.0 and 10.7.1, the query becomes a lot slower and frequently stuck with error below:

@firebase/firestore: Firestore (10.7.0): WebChannelConnection RPC 'Listen' stream 0x5b9a037f transport errored: Wn {type: 'c', target: Hn, g: Hn, defaultPrevented: false, status: 1}

Switch back to 10.6.0 and the query completes quickly.

Steps and code to reproduce issue

I've created a minimal sample to reproduce this issue and have shared with @MarkDuckworth, if you need to get access to the private repo, please let me know, thank you!

[ehsannas]

Thanks for reporting @thomasdao. I'll try to reproduce it

[thomasdao]

@ehsannas thanks, I've invited you to the sample project :)

[ehsannas]

Thanks @thomasdao . I am able to see the error in the logs from your repo. I do, however, see that each such log message is followed by an UNAVAILABLE code from the backend. Which means it's a legitimate error returned from the backend to the SDK. It's plausible that the newer WebChannel version has become much more efficient at sending parallel requests to the backend such that you're hitting a certain limit of request rate for a single client. This error code is retryable with a backoff, which means the SDK will recover and rerun the query after some delay.

Please take a look at:
https://firebase.google.com/docs/firestore/real-time_queries_at_scale#understand_high_write_traffic_in_the_system
https://firebase.google.com/docs/firestore/best-practices#ramping_up_traffic

[thomasdao]

@ehsannas I've never seen the UNAVAILABLE code, even if I wait for more than 10 minutes.

I find the reason newer WebChannel version has become much more efficient at sending parallel requests not really logical: the same type of query works with version 10.6.0, which indicates that the server is able to handle that query and the problem is likely with the newer version of the client.

I've tested adding a delay of 1 second between each paginated query to reduce server load, and see the same error @firebase/firestore: Firestore (10.7.0): WebChannelConnection RPC 'Listen' stream 0x269fb953 transport errored: Wn {type: 'c', target: Hn, g: Hn, defaultPrevented: false, status: 1}.

[phileasthefogg]

I'm also running into this error. Subscription seems to work fine for a while and then gets dropped with the same RPC 'Listen' stream transport error. Any ideas on what this might be or where to catch the error?

[IvanKYW]

Same issue after upgrade AngularFire to 17.0.1 which depends on firebase ^10.7.0.

One of our project query becomes slower and run into the @firebase/firestore: Firestore (10.7.2): WebChannelConnection RPC 'Listen' stream error occasionally. The other smaller project works fine.

Tried experimentalForceLongPolling mentioned in #7968 but no luck. downgrade to 10.6.0 seems resolve the issue.

[ghinda]

I'm also seeing the same issue with hanging snapshot queries for a while, with the same type of WebChannelConnection RPC 'Listen' stream ... transport error.

Sometimes, after failing with the error, the snapshot query retries and returns correct data after a couple of minutes, but most times it just hangs indefinitely. In our case, it only happens with queries that would return a large amount of data (hundreds of docs containing fairly large strings).

The issues started with versions 10.4.x. They were then fixed in versions 10.6.x, but are now back again with 10.7.x. I've also tested the latest 10.8.0, and the issue is still there. As a summary:

• 10.3.1: issue not present
• 10.4.x: issue shows up
• 10.5.x: issue still present
• 10.6.x: issue fixed
• 10.7.x / 10.8.0: issue shows up again

Using experimentalForceLongPolling does not seem to make a difference.

I wasn't able to reproduce it in a local or staging environment, as it only seems to show up in our production environment where we have around ~40K snapshot listeners / ~10K active connections, as reported in the Firebase console.