Module cache keying semantics

[guybedford]

Currently the module cache is typically keyed uniquely by URL.

With this proposal the module identity now seems to include both the URL and the attributes.

Would the expectation be that the module cache keying would be extended to support the attribute identity?

Or alternatively will the URL remain the primary keying with some reconciliation approaches in place? How would those approaches avoid causing indeterminism based on load ordering?

[devsnek]

To be more specific, modules in the spec are keyed as (referrer, specifier) pairs, so the exact problem is a single file with two imports of the same specifier with different attributes.

[littledan]

I was imagining that the module attributes would not be part of the key. That is, the same module specifier and referrer would always point to the same module record, even when imported twice with different parameters. However, a subsequent load with different module attributes may lead to an exception being thrown rather than the module returned. Here's some examples of possible semantics (each of which could be debated further) which are all consistent with the idea that the module attributes are not part of the key, but the host may keep them around:

• With type on the web, if a module is imported once, the type must either be provided or missing (JS), and the subsequent loads must pass the same type option (otherwise the check would've failed).
• If we have fetch options passed through module attributes, subsequent loads could either repeat the fetch options of the initial load, or omit them, but not pass additional or different options.
• If we have signature-based SRI passed through module attributes, any site for importing the module may pass in a signature, and that that would be checked against the module contents for that import site (that is, SRI errors would not be cached, in this idea).

This sort of interpretation rules out some use cases that @xtuc mentioned about module attributes determining the processing mode, when used in a way that a module could be imported multiple times in different processing modes.

[bmeck]

It seems like races could occur if first to import wins:

try {
  // try to load as JS, this could make all attempts to load JSON fail
  await import('foo');
} catch (e) {
  // try to load as JSON? maybe thats how other people are using this thing?
  const {default: foo} = await import('foo', {type: 'json'});
  // clobber stuff
  Object.defineProperty(foo, 'bar', {
    get() {
       // fun stuff like arguments.caller / Error.prepareStackTrace / etc. go here
    },
    set(v) {
       // more clobbering/calling here
    }
  });
}

I have serious reservations about the viability of keeping everything in sync across all call sites and if throwing on mismatch is desirable. It seems like throw on mismatch could lead to just iterating all the possible values in order and doing things as if it were some kind of odd overloading technique.

[littledan]

@bmeck This is an interesting example. Sounds like we probably shouldn't cache failure due to module type check mismatch.

[hax]

So allow retry when type mismatch? But current import('foo') will not allow retry if type is not js mimetype.

[jkrems]

In terms of use cases, this points to two things being fundamentally at odds (based on some not-so-theoretical host that acts like a browser):

1. Module attributes are optional, allowing for incremental adoption of an attribute (assertion-style attributes). This implies that module identity doesn't depend on attributes.
2. Module attributes change the interpretation of the URL, e.g. by adding or modifying headers or by changing how the fetched resource is evaluated. This implies that module identity does depend on attributes.

It would feel pretty unfortunate if those two semantics would mix. Not only would tools and users have to have perfect knowledge on the cache semantics of each individual attribute (as opposed to module attributes in general), it would also affect the ability to support unknown attributes. A host couldn't just ignore an unknown attribute because it would have ambiguous cache key semantics.

And unless new module attributes can only ever apply to module types that have never been supported before, (1) seems pretty important. Otherwise it would require global coordination across files to make sure that the same target is never requested with different attributes.

P.S.: Global coordination because this is speculating about likely behavior in a browser-like host where there's a global/realm-wide module map that has some caching semantics. From a JS spec perspective it would only be per-importing-file coordination. But I don't think browsers would pick global caching semantics that are incompatible with the per-file ones. So - effectively it's global cache-by-URL that most users would observe.

[Jack-Works]

We should split module attributes that will and won't change the identity of the module, it's better on the syntax level.

For example:

These two modules are different

import mod from '/mod' with { type: 'css' }
import mod1 from '/mod' with { }

These two modules are the same

import mod from '/mod' with {} and { integrity: 'hash-of-this-file' }
//                          ~~     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
import mod1 from '/mod' with {} and { cache: '1year' }

In the first attribute object {}, any difference in it cause them to be two different modules, but in the second attribute object, module with different attribute object are treated as the same module.

But this is not enough, we need a syntax for the developer to specify the following orthogonal behavior:

• Identity: If this attribute changes, should it be the same module? (For type, no. For integrity, yes)
• Ignorable: Is the attribute ignorable if the runtime doesn't know it's semantics?
• ...: any more?

[littledan]

@Jack-Works I see type as being in the same category as integrity: they are both checks. type always checks a type which is present in some other way, e.g., the MIME type or file extension. (That said, there are some significant arguments against having integrity in particular.) These would not be part of the cache key (failure vs non-failure would differ based on what's passed, but they couldn't have multiple distinct success values.)

There's another category which would change the interpretation of a module. For example, parameters to JSON.parse, or interpreting the path as a string that's default-exported rather than what it would be otherwise. These sorts of module attributes would need to be part of the cache key.

[Jack-Works]

@littledan I don't think type is a simple check. The browser can send different sec-fetch-dest header based on the value the type therefore server can return different formats based on the sec-fetch-dest header.
Therefore, files with different type must be treated as different modules. But if I import a module with integrity and import it again without integrity, I expect it to be the same module

[littledan]

@Jack-Works Let's discuss this further in #24 . We'll need a specification for what browsers do exactly before Stage 3, and web specs like fetch tend to go into that level of detail.

[Jack-Works]

@littledan oh, so type is just my example, I really want to emphasize is that we should have a signal to tell the runtime if the attribute is ignorable or will affect the identity of the module even they don't know what the attribute is.

[jkrems]

I like that separation! But the and feels awkward because it favors cache-busting attributes over ones that maintain a single module per URL. Maybe something like this would express the same:

// Attributes that only assert properties on the resource but don't affect how it's fetched:
import mod from '/mod' assert { integrity: 'hash-of-this-file' }
import mod from '/mod' expect { integrity: 'hash-of-this-file' }
import mod from '/mod' assert { type: 'json' }

// Attributes that may affected how the module is being fetched:
import mod from '/mod' with { credentials: 'include' }

// Combined
import mod from '/mod' with { credentials: 'include' } assert { integrity: 'hash-of-this-file' }

Therefore, files with different type must be treated as different modules.

I don't think that's necessarily true. One big downside is that type could never be optional which is a bit unfortunate. Having a non-optional type attribute means there's a big pressure to only ever load modules from JavaScript modules (to make things easier to use / less noisy). So even endpoints that could've been simple data files would be returning executable scripts instead.

[littledan]

I continue to have the understanding that type would be required in the web, and non-web environments could decide whether to require it or make it optional.

[hax]

I continue to have the understanding that type would be required in the web, and non-web environments could decide whether to require it or make it optional.

So it would make the modules env-specific just because json/wasm modules?