Skip to content

Fix error in Memory LRU implementation #1299

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Oct 10, 2018
Merged

Conversation

gsoltis
Copy link

@gsoltis gsoltis commented Oct 9, 2018

Iterating orphaned documents was returning all documents, not just orphaned ones. They weren't being incorrectly GC'd, but they were incorrectly counting towards sequence numbers to GC.

The test that caught this is coming in the port of firebase/firebase-ios-sdk#1905

Additionally, I implemented Symbol.iterator for ObjectMap so that I could use PersistencePromise.forEach().

@@ -17,7 +17,7 @@
import { Equatable } from './misc';
import * as objUtil from './obj';

type Entry<K, V> = [K, V];
export type Entry<K, V> = [K, V];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already export a type Entry in sorted_map.ts. I would actually prefer if we use that type since it's easier to digest (it's an object on not an array)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has to be exported because it's used in the constructor for ObjectMapIterator, it's not actually anywhere in the public api for ObjectMap or the iterator though. I would prefer not to add a dependency from this collection to SortedMap, since they aren't really related.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we do keep it, then please rename it. Having two Entry types that are very similar will make auto-importing a bit tricky.

[Symbol.iterator](): Iterator<{ key: KeyType; value: ValueType }> {
// We don't care about the keys, all of the actual keys are in the
// arrays that are the values of the inner object.
const it = objUtil.values(this.inner)[Symbol.iterator]();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This creates a copy of the value array, which is a little surprising for an iterator. I don't know a good way around this without using generators (which I assume we can't), so I am going to ignore this.

I do think this could be simplified quite a bit though. Wouldn't this work just as well:

    const values : Array<{ key: KeyType; value: ValueType }> = [];
    this.forEach((key, value) => values.push({key, value}));
    return values[Symbol.iterator]();

You might not need the ObjectMapIterator.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, I don't know a way to avoid creating the array of values. Objects don't support iterators by default, and there's no ordering either to be able to start and stop iteration through the object.

As for using forEach, it kinda depends on whether or not we think there are collisions. ObjectMapIterator doesn't need to create all of the entries that will be returned at once. That's only a benefit though if there are collisions, since we do need the values array.

I'm not sure I have a strong opinion either way, since it depends on what's stored. Creating the whole array up front does have the advantage of not needing to export an extra Entry type, I suppose.

Do you feel strongly either way?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The two solutions (copy every value and copy every set of matching values) have such different complexities that I would much prefer to pick the simpler one.
We can look at the usage in the SDK and see how common collisions are, but from memory it seems that they should be pretty rare (my IDE doesn't open at the moment...). The overhead from an occasional canonical ID collision can probably be ignored.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I went with the forEach method.

/**
* Implements an Iterator over ObjectMap.
*/
export type IteratorEntry<KeyType, ValueType> = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, This is the Entry type from SortedMap.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, but I don't think a dependency between them is appropriate.

@gsoltis gsoltis assigned schmidt-sebastian and unassigned gsoltis Oct 9, 2018
Copy link
Contributor

@schmidt-sebastian schmidt-sebastian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@gsoltis gsoltis merged commit fa9c3e5 into master Oct 10, 2018
@gsoltis gsoltis deleted the gsoltis/fix_memory_lru branch October 10, 2018 02:34
@firebase firebase locked and limited conversation to collaborators Oct 15, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants