Requesting data with source: server does not actually go to server and returns data from cache

[andreymikhadyuk]

Operating System

Windows 11 Pro 22H2

Browser Version

Firefox v120.0.1 (64-bit)

Firebase SDK Version

^8.3.2

Firebase SDK Product:

Firestore

Describe your project's tooling

It is web app written using React v17.0.1. For state management we use redux (with sagas).

Describe the problem

We've enabled persistence in our app. In some places we rely on the cached data. So sometimes we do request using source: cache and if there is no data in cache we try same request using source: server.

We display a list of items and for each item we fetch its discussion. The issue is that for one of our teammates somehow after some time of app usage some discussions become empty in the cache, but even doing a request with source: server does not really go to server, we still get empty data in the response (and yes, by the query we have data in the db). In another browser or after clearing the cache the issue is solved. Changing cache size to unlimited did not solve the problem.

Me and my teammate tried to intentionally reproduce the issue by doing following:

• we tried to limit the cache size to 1 Mb and play with the app. We tried to reach some cache clearings by garbage collector and everything worked fine, we could not get the state when we see incorrect data (I tested in Chrome (119.0.6045.159) and Firefox (120.0.1) on my MacBook (macOS 13.4.1));
• going offline with source: server did not throw an error. The code continued working in the try block and in the logs we could see null data having fromCache=false.

So we could not reproduce the issue. But my teammate has this issue in our dev and prod environments in his Firefox.

Steps and code to reproduce issue

As I mentioned in the problem description, we could not intentionally reproduce it.
Here is the code example of how we fetch that data:

const getDiscussionById = async (
  discussionId: string,
  source: "default" | "server" | "cache" = "default",
): Promise<Discussion | null> => {
  try {
    const snapshot = await firebase
      .firestore()
      .collection("discussion")
      .withConverter(converter)
      .doc(discussionId)
      .get({ source });
    const discussion = snapshot.data() || null;

    if (!discussion && source === "cache") {
      return getDiscussionById(discussionId, "server");
    }

    return discussion;
  } catch (error) {
    if (source === "cache" && isFirestoreCacheError(error)) {
      return getDiscussionById(discussionId, "server");
    } else {
      throw error;
    }
  }
};

We tried to remove the logic of fetching with source: cache by using just source: server to get data directly from server, but we still get empty data having fromCache=false. As I mentioned in the problem description, going offline does not throw the error during the request, the code keeps executing in the try block with empty data in the snapshot.data().

const getDiscussionById = async (
  discussionId: string,
): Promise<Discussion | null> => {
  const snapshot = await firebase
    .firestore()
    .collection("discussion")
    .withConverter(converter)
    .doc(discussionId)
    .get({ source: "server" });
  const discussion = snapshot.data() || null;

  return discussion;
};

[ehsannas]

Hi @jbalidiong , thanks for reporting.

You seem to be using a very old SDK version (8.3.2 is at least 2 major versions behind). The newest version is 10.7.1. Please upgrade to the newest version. Unfortunately we won't look into issues related to version 8, as it's been deprecated now.

If you continue to see the issue in the latest version, please let me know along with a way to reproduce the issue.

[google-oss-bot]

Hey @andreymikhadyuk. We need more information to resolve this issue but there hasn't been an update in 5 weekdays. I'm marking the issue as stale and if there are no new updates in the next 5 days I will close it automatically.

If you have more information that will help us get to the bottom of this, just add a comment!

[andreymikhadyuk]

@ehsannas Thanks for the info, I've updated the version and we will check whether the issue is still there or not.

[andreymikhadyuk]

After updating the version to latest one we still had fully the same issue. We decided to clear the cache and see whether the issue will return or not with the latest version. If yes, then I will come back to this issue.

[andreymikhadyuk]

@ehsannas Hello!
We've gotten the issue again. Now in Chrome 121.0.6167.140 (64-bit) on the same computer. Firebase SDK Version is ^10.7.1 now.
The issue is the same:

1. we try to fetch specific data using source: cache
2. after getting null we try to make request using source: server and it also returns null (the query is correct, the data is in the DB and in another browser for the same user everything is fine)

Here is the snapshot screen (you can see that fromCache is false and the response is empty):

I do not really have steps to reproduce, it is such a rare thing, but we cannot fix it. The only way is to clear the cache in the browser.
Is there a potential fix on our side or on your side?

[DanielFlockhart]

@ehsannas Hello! We've gotten the issue again. Now in Chrome 121.0.6167.140 (64-bit) on the same computer. Firebase SDK Version is ^10.7.1 now. The issue is the same:

1. we try to fetch specific data using source: cache
2. after getting null we try to make request using source: server and it also returns null (the query is correct, the data is in the DB and in another browser for the same user everything is fine)

Here is the snapshot screen (you can see that fromCache is false and the response is empty):

I do not really have steps to reproduce, it is such a rare thing, but we cannot fix it. The only way is to clear the cache in the browser. Is there a potential fix on our side or on your side?

Any Updates on this? Dealing with the same problem.

[andreymikhadyuk]

@ehsannas Hey👋🏻 Any thoughts on this?

[ehsannas]

hi folks, can anyone provide a minimal code to reproduce this?

[andreymikhadyuk]

hi folks, can anyone provide a minimal code to reproduce this?

From my side this is just a rare thing which appears after some time of using our app.
@DanielFlockhart Maybe you know how to reproduce this issue?

[noskiboots]

hi folks, can anyone provide a minimal code to reproduce this?

From my side this is just a rare thing which appears after some time of using our app. @DanielFlockhart Maybe you know how to reproduce this issue?

Any updates? Same exact issue in my app. Only every once in a while and specifically in chrome Version 123.0.6312.107 (Official Build) (x86_64).

[dconeybe]

Drive-by comment: If anyone is able to reproduce this bug, please enable Firestore debug logging by calling setLogLevel('debug') (docs) and providing the logs. The logs will give some insight into the internals of the SDK and the steps it is taking to surface the results of the query, which may help us debug in the absence of concrete reproduction steps.

[bbonds007]

Can confirm I am experiencing the same exact issue. Its been plaguing my firebase project for almost a year now and I only now have been have to locate the issue due it being such a rare and difficult bug to locate.

Currently on Firebase ^10.8.1

I haven't tried Firestore debug logging yet but I will attempt that tomorrow and report back if I find anything.

Here are the bullet points of what I found so far.

• Only appears to effect Chrome, firefox appears unaffected even when trying to replicate it with the steps below.
• Only effected if firebase persistence is enabled (if you turn persistence back off, then .get() retrieves the document just fine)
• I can somewhat reliably reproduce the bug. While I have had it appear organically, its very rare and partially deleting cookies is the fastest way to replicate it.

1. Start by clearing my "Cookies and other site data" entirely by starting fresh.
2. Run my project and sign in using firebase auth, and be sure to retrieve a few documents from firestore.
3. Wait OVER 1 hour at least, then go under chrome settings -> "Clear browsing data" -> then delete "Cookies and other site data" for ONLY the last hour. This effectively deletes part of my IndexedDB files, and I believe is the source of the problem. I believe some IndexedDB documents are being deleted and labeled as "noDocument". When this happens, it then breaks firebase's ability to ever retrieve that document again, even if .get({ source: 'server' }) is used, it will still return the doc.exists: false.

Temporary Solution: This isn't a great solution but it can fix the issue if you are able to detect it occurred. Basically just delete IndexedDB and reload the page so it does a fresh reload of firestore and its local cached database.

1. await this.AngularFirestore.firestore.terminate()
2. await this.AngularFirestore.firestore.clearPersistence()
3. window.location.reload();

[bbonds007]

Drive-by comment: If anyone is able to reproduce this bug, please enable Firestore debug logging by calling setLogLevel('debug') (docs) and providing the logs. The logs will give some insight into the internals of the SDK and the steps it is taking to surface the results of the query, which may help us debug in the absence of concrete reproduction steps.

@ehsannas @dconeybe

Okay here are my Firestore debug logs. Hopefully its helpful, I apologize if they are lengthy, but there does appear to be some important happenings in them, specifically the problematic document in question (users/i8AVk3h2EcdBt2SC9fXmimOE89A2) appears to be getting deleted in the 2nd to last screenshot.

[dconeybe]

For anyone experiencing this issue, please upgrade to v10.12.1 (released May 20, 2024) and try again. A fix went into this version (#8247) for a bug where flaky/empty snapshots were being raised when multi-tab persistence was enabled. I have high confidence that the problems reported in this issue will be fixed in that version.

[google-oss-bot]

Hey @andreymikhadyuk. We need more information to resolve this issue but there hasn't been an update in 5 weekdays. I'm marking the issue as stale and if there are no new updates in the next 5 days I will close it automatically.

If you have more information that will help us get to the bottom of this, just add a comment!

[google-oss-bot]

Since there haven't been any recent updates here, I am going to close this issue.

@andreymikhadyuk if you're still experiencing this problem and want to continue the discussion just leave a comment here and we are happy to re-open this.