Key Paths: Support iteration step operator

[inexorabletash]

Proposed in https://www.w3.org/Bugs/Public/show_bug.cgi?id=28456 by [email protected]:

In one of our apps I would like to store objects like this:
{ children: [{ id: 'c1' }, { id: 'c1' }] }

I would like to be able to create an index to quickly find all parents for child 'c1'.

Sadly an index like "children.id" { multiEntry: true } does not work here.
I would propose something like "children[].id" { multiEntry: true } to indicate, that the children property should be iterated, not their id field.

Related discussions:
https://social.msdn.microsoft.com/Forums/en-US/dbf2fd2d-a8ca-4228-8a49-3948d44f7e26/indexeddb-and-complex-keypaths-for-index (where Israel Hilerio (MSFT) mentions this should be discussed for IDB v2)
dexie/Dexie.js#86

[inexorabletash]

Example: http://stackoverflow.com/questions/36034150/indexeddb-multyentry-with-sub-objects

[inexorabletash]

Example: http://stackoverflow.com/questions/36921561/indexing-array-values-in-an-object-in-an-indexeddb

[inexorabletash]

What are the associated edge cases if we allowed the proposed syntax: "a.b[].c" ? What does the grammar look like? Is "a[][]" valid? (yes?) Is "[]" valid? (yes?) Is "a[]a" valid? (no?)

path := "" | (identifier | iteration) step*
step ::= "." identifier | iteration 
iteration ::= "[]"
identifier ::= IdentifierName /* [ECMA-262] */

Parsing then leads to a sequence of (identifier | iteration); https://w3c.github.io/IndexedDB/#dfn-evaluate-a-key-path-on-a-value would need to parse (not just split on "." then iterate), and if an iteration step is hit uses GetMethod(v, @@iterator) / GetIterator() like WebIDL and iterates/recurses, producing a key array.

[luxlux]

the lack of this feature makes using indexDB really no pleasure

[Dwyguy]

It's been a while, but any movement or plans on this? I just encountered a similar issue, and it would be nice.

[MidgetAteMyMom]

push

[maxgaurav]

Any update on this?

Would love to have index creation using something like property.*.arrayed_object_property_name

[inexorabletash]

Concrete proposals would be welcome to solicit feedback here, and implementer interest could be solicited. (If the right attendees are there, we can bring this up at TPAC, but no need to wait for F2F.)

https://youtu.be/y1TEPQz02iU might be worth watching...

[asutherland]

As Firefox (Gecko) has been cleaning up our IndexedDB (key) implementation, we've been talking with our JS engine (SpiderMonkey) team, and they would really like it if IndexedDB's ECMAScript Binding stuff at https://w3c.github.io/IndexedDB/#binding actually lived in (or closer to) the ECMAScript spec and our (Gecko's) implementation of those hooks lived in the JS engine (SpiderMonkey).

So I think Firefox's position right now is that we wouldn't want to change any of the binding stuff until we've tried to pursue that. We'd certainly be interested to know if other IndexedDB implementers' JS teams are similarly inclined.

[inexorabletash]

I followed up a bit with our (Chrome's) JS team. It's not seen as a priority, and we don't feel that having the spec and implementation partitioned in the same way is necessary for implementation improvements. As an example, we migrated our structured clone logic from Blink to V8 to get a bit speed-up, no spec changes required. On the flip side, I'm certainly not married to maintaining this complex bit of spec text if someone else wants to adopt it.

Someone also mentioned that even if there is a desire to re-use the property lookup logic elsewhere (e.g. expose for userspace libraries), optional chaining/null coalescing is going to change simplify writing similar lookups directly in script. Not sure if that was part of the thinking by the SpiderMonkey team, though.

[fdinburgh]

the ticket is 7 years old and open, any progress? indexing fields of objects inside array is a real desired feature i believe

[4e576rt8uh9ij9okp]

I understand that the specifications must be solid and thought over, but 8 years?

Specifications goal should be to add the MOST USED things and functions into the specifications so people don't need to hack around but code instead.

Even after 8 years, YES WE NEED IT and this won't go away because we just need it.

[1toddlewis]

Adding another voice and nearly six months of time to this ongoing request. The lack of this feature is really hampering the utility of IndexedDB. There has to be a way to implement this.

Dexie uses an asterisk to denote a "multi-index." MongoDB uses a dollar sign as a placeholder for the iterator. So how about merging those ideas and using *x.$.y to designate x is an array of objects that have an index y. It's time to resume the conversation!

[DadvDadv]

Azure Cosmos DB use this notation :

{
    "path": "/compositions/[]/id/*"
 }

The [] is replace by the array index on the fly.