Skip to content

Is this proposal strict enough for its use cases? #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
andreubotella opened this issue Apr 10, 2024 · 15 comments
Open

Is this proposal strict enough for its use cases? #1

andreubotella opened this issue Apr 10, 2024 · 15 comments

Comments

@andreubotella
Copy link
Member

andreubotella commented Apr 10, 2024

This proposal's name is "strict enforcement of using", but while it solves the footgun of forgetting using, it is not strict against unintended usage of the disposable API.

You might expect that a strict enforcement of an RAII-like pattern would guarantee not only that the [Symbol.dispose] method is always called, but that it is called before the current function returns, and that the ordering of [Symbol.enter] and [Symbol.dispose] calls across multiple disposables is properly stacked. Since this proposal allows JS code to call the [Symbol.enter]() and [Symbol.dispose]() methods, it guarantees none of these properties.

As an example use case where these properties matter, in the AsyncContext proposal we were discussing the possibility of a disposable API (see tc39/proposal-async-context#60 and nodejs/node#52065), that would switch the current context when entered and restore the previous one when disposed. However, since the AsyncContext machinery is unaware of function boundaries, the possibility of an entered but non-disposed scope would pollute the caller's context:

import { callee } from "some-library";

const asyncVar = new AsyncContext.Variable();

function caller() {
  asyncVar.run(42, () => {
    console.log(asyncVar.get());  // 42

    callee();

    // As the AsyncContext proposal currently stands, there would be nothing
    // callee can do that can make asyncVar.get() not return 42. However...
    console.log(asyncVar.get());  // undefined
  });
}
// some-library

// The AsyncContext snapshot at the time this module was evaluated, which is
// likely empty.
const initialSnapshot = new AsyncContext.Snapshot();

export function callee() {
  const scope = new AsyncContextScope(initialSnapshot);
  scope[Symbol.enter]();
}

Now, it is possible that this is a one-off, or a special case, and that most disposables would benefit from fixing the footgun without needing this level of strictness. But that would need to be looked into.

@ggoodman
Copy link

ggoodman commented Apr 10, 2024

At first I was trying to think through some approach wherein some clever indirection could add some stronger guarantees to these Resources. I was thinking, "what if Symbol.enter() received an argument from the runtime with which the Resource author can register the instance?" But then I realized that this would just add another hoop to jump through without actually preventing the foot-gun.

I couldn't think of many precedents in terms of APIs that strictly required runtime support like this. The obvious one that did come to mind though, was Promise and the await keyword. I think this is a relevant precedent in that the API was designed in a way that had built-in foot guns like: await new Promise(() => {});. The success of Promise and async / await despite these weaknesses suggests that perhaps it's reasonable to compromise when it allows for user-land shims and transpilation solutions to exist.

@rbuckton
Copy link
Collaborator

rbuckton commented Apr 10, 2024

it is not strict against unintended usage of the disposable API.

Our position is that if you are explicitly calling [Symbol.enter](), you are intentionally opting in to doing everything yourself. It is imperative that there is a manual way to interact with disposables if you want to build your own resource management building blocks on top of disposables. [Symbol.enter]() is the way you do that.

You might expect that a strict enforcement of an RAII-like pattern would guarantee not only that the [Symbol.dispose] method is always called, but that it is called before the current function returns, and that the ordering of [Symbol.enter] and [Symbol.dispose] calls across multiple disposables is properly stacked. Since this proposal allows JS code to call the Symbol.enter and Symbol.dispose methods, it guarantees none of these properties.

That is explicitly not a feature of this proposal and I am not making any such guarantees. Symbol.enter is intended purely as a stumbling block to guide people to using. I do not intend it to have any other power or capability over and above that. If you explicitly call [Symbol.enter](), you are expected to know what you are doing. If that means renaming the symbol to Symbol.enter_DO_NOT_USE_OR_YOU_WILL_BE_FIRED or something similar but less meme-y to get the point across, I'm perfectly fine with that.

However, since the AsyncContext machinery is unaware of function boundaries, the possibility of an entered but non-disposed scope would pollute the caller's context

Sure, but if you've chosen to explicitly call [Symbol.enter]() without manually handling cleanup, then you've written buggy code.

@andreubotella
Copy link
Member Author

That is explicitly not a feature of this proposal and I am not making any such guarantees. Symbol.enter is intended purely as a stumbling block to guide people to using. I do not intend it to have any other power or capability over and above that. If you explicitly call [Symbol.enter](), you are expected to know what you are doing. If that means renaming the symbol to Symbol.enter_DO_NOT_USE_OR_YOU_WILL_BE_FIRED or something similar but less meme-y to get the point across, I'm perfectly fine with that.

I think the name [Symbol.enter] might be fine, but "strict enforcement" as the name of the proposal might suggest the wrong thing.

@jridgewell
Copy link
Member

Our position is that if you are explicitly calling [Symbol.enter](), you are intentionally opting in to doing everything yourself. It is imperative that there is a manual way to interact with disposables if you want to build your own resource management building blocks on top of disposables. [Symbol.enter]() is the way you do that.

Not arguing against that, but we are saying that it's not a strict enough guarantee to solve AsyncContext's problem. We'd like to explore a way to do both, so that other APIs that care about strict enforcement are able to implement it.

Sure, but if you've chosen to explicitly call [Symbol.enter]() without manually handling cleanup, then you've written buggy code.

It's not just manual calls to scope[Symbol.enter]() that are a problem. Even DisposableStack if you were to properly register it still allows an accidental leak that we'd like to prevent.

class Foo {
  private stack = new DisposableStack();

  example() {
    const scope = new AsyncContext.Scope(someSnapshot);
    disposable.use(scope);

    // Do stuff…

    // didn't dispose here!
  }

  [Symbol.dispose]() {
    this.stack[Symbol.dispose]();
  }
}


const v = new AsyncContext.Variable();
using f = new Foo();

v.run(1 , () => {
  assert.equal(v.get(), 1);

  // f is properly using a DispoableStack, but that's not enough
  // to prevent a leak.
  f.example();

  // f's context has leaked out of it's method call.
  assert.equal(v.get(), undefined);
});

Ideally, we'd like a way to detect syntatic using, because that's the only thing that guarantees our ability to restore the async context state to what it was prior to the mutation.

@shicks
Copy link

shicks commented Apr 12, 2024

@jridgewell

Ideally, we'd like a way to detect syntatic using, because that's the only thing that guarantees our ability to restore the async context state to what it was prior to the mutation.

This seems problematic w.r.t. syntax transformation. This sort of feature would (by definition) be untranspilable, making it useful (at best) only in the very-narrow case of transpiling later language features that would depend on it.

@ljharb
Copy link
Member

ljharb commented Apr 12, 2024

Such a mechanism would have to also support eg const s = new DisposableStack(); s.use(f); using s;, and I'm not sure how that would be possible in a reliable fashion.

@jridgewell
Copy link
Member

@shicks

This seems problematic w.r.t. syntax transformation. This sort of feature would (by definition) be untranspilable, making it useful (at best) only in the very-narrow case of transpiling later language features that would depend on it.

Someone mentioned a function.using meta property in Matrix, and that would work fine and be polyfillable with minimal transform. Just inect a global, private boolean, set it to true immediately before using, and set it back to false immediately after.


@ljharb

Such a mechanism would have to also support eg const s = new DisposableStack(); s.use(f); using s;, and I'm not sure how that would be possible in a reliable fashion.

During the using s, the DisposableStack[Symbol.enter] method would be able to detect it's in a syntatic using. But the original f cannot, because it's already been entered by the s.use(f) call.

@ljharb
Copy link
Member

ljharb commented Apr 12, 2024

Right - which is why "enforcing syntactic usage" won't work, because that would give you a false negative.

@mhofman
Copy link
Member

mhofman commented Apr 12, 2024

Someone mentioned a function.using meta property in Matrix, and that would work fine and be polyfillable with minimal transform. Just inect a global, private boolean, set it to true immediately before using, and set it back to false immediately after.

Not quite. It would have to be reset for further nested stacks.

@jridgewell
Copy link
Member

jridgewell commented Apr 12, 2024

@ljharb

Right - which is why "enforcing syntactic usage" won't work, because that would give you a false negative.

Not from my point of view. I specifically do not want it to pass, so it's giving a true negative. It is not acceptable to use AsyncContext within a DisposableStack.


@mhofman

It would have to be reset for further nested stacks.

Ah, yes, it should reset to the previous value. But still easily implementable with a transform.

@ljharb
Copy link
Member

ljharb commented Apr 12, 2024

Why isn't that acceptable? It's going to be properly disposed.

This really feels like it should just be a linter rule, like no-floating-promises - especially given that there's differing opinions about what level of enforcement is desired.

@rbuckton
Copy link
Collaborator

@jridgewell and I have been discussing this on Matrix. The kinds of guarantees that AsyncContext.Scope needs would preclude it from being used in a DisposableStack and would prohibit any kind of composition. I'm concerned that a carve out for this case could poison adoption in the ecosystem as users will have less confidence about whether a resource can be used with using or DisposableStack interchangeably, or whether refactoring from

using x = new Resource();

to

const getResource = () => new Resource();
using x = getResource();

will break their code.

The intended use of AsyncContext.Scope does overlap with using, so it makes sense that they would seek to employ using for that purpose. However, given the potential impact on adoption as a result of user confusion, I'd prefer if the more specific needs of AsyncContext.Scope be handled by way of a syntactic opt-in as it would give users more confidence in making a determination as to whether simple refactors will be safe, e.g., using scope(ctx), or something to that effect.

I have also proposed an alternative approach, where each successive new Scope is maintained in a stack, such that when AsyncContext.run exits it would clean up any non-exited Scope objects and then throw. This would provide feedback to the developer about an incorrect use of the feature without necessitating a special case in using, but that did not seem to sufficiently address their concerns.

@dead-claudia
Copy link

I like the function.using idea, but I feel that a Symbol.enter is unnecessary for enforcement if you forward a function.using correctly and add a return using syntax for semantically significant tail calls.

Here's the idea: set function.using to true in the following situations, false everywhere else.

  • In the resulting method call of using x = func().
  • In the resulting method call of using x = o.method(), but not in the o.method property access itself.
  • In the resulting getter or proxy handler call of using x = o.key.
  • In the resulting method call of return using func(), when function.using is set to true in the outer function.
  • In the resulting method call of return using o.method(), when function.using is set to true in the outer function, but not in the o.method property access itself.
  • In the resulting getter or proxy handler call of return using o.key, when function.using is set to true in the outer function.

function.usingAwait would be defined similarly for using await declarations. Both would use the same return using syntax.

In both cases, no new methods need to exist, and you get an unforgeable property you can rely on.


This plus two more other optional methods makes async variables entirely implementable in userland, handling assignment through used resources:

function.using is also set in the body of each of these when called at the language level.

  • handle = resource[Symbol.fork](): Called to fork the resource for subsequent entry. Returns an optional handle to restore state with.
    • If missing, nothing gets registered on the generator. Also, nullish return values are tolerated and also cause nothing to be restored.
    • This may be called at any time.
    • If present at using time, it's called at the language level for generator construction, yield, yield*, and await. It's saved as part of the using statement's execution, so implementors can maintain smaller snapshot states.
  • resource = handle[Symbol.enter](): Called after internal generator resume to enter a handle and re-acquire the resource. function.using is set to true in the body of this as well.

Async context could then be implemented like this:

Click to expand
let setCurrent

class VariableFork {
    #key, #value

    constructor(key, value) {
        this.#key = key
        this.#value = value
    }

    [Symbol.enter]() {
        return this.#key.withValue(this.#value)
    }
}

class VariableSet {
    #key, #value, #old
    #disposed = false

    constructor(key, value, old) {
        this.#key = key
        this.#value = value
        this.#old = old
    }

    [Symbol.fork]() {
        return new VariableFork(this.#key, this.#value)
    }

    [Symbol.dispose]() {
        if (this.#disposed) return
        this.#disposed = true
        setCurrent(this.#key, this.#old)
        this.#old = undefined
    }
}

delete VariableFork.prototype.constructor
delete VariableSet.prototype.constructor

export class Variable {
    #current, #name

    static {
        setCurrent = (v, c) => {
            v.#current = c
        }
    }

    constructor(opts = {}) {
        this.#current = opts.default
        this.#name = opts.name ?? ""
    }

    get name() {
        return this.#name
    }

    get() {
        return this.#current
    }

    withValue(value) {
        if (!function.using) {
            throw new TypeError("This must only be called with `using`")
        }
        const old = this.#current
        this.#current = value
        return new VariableSet(this, value, old)
    }

    run(value, f, ...args) {
        using _ = this.withValue(value)
        return using f(...args)
    }
}

function *snapshotExecutor() {
    let value
    while (true) {
        const {f, args} = yield value
        value = f(...args)
    }
}

export class Snapshot {
    #active = new Variable(false)
    #executor = snapshotExecutor()

    constructor() {
        this.#executor.next()
    }

    run(f, ...args) {
        if (this.#active) return f(...args)
        using _ = this.#active.withValue(true)
        return this.#executor.next({f, args}).value
    }

    wrap(f) {
        return (...args) => this.run(f, ...args)
    }
}

@shicks
Copy link

shicks commented Apr 21, 2025

@dead-claudia That's an interesting and clever idea, but I'm unclear on how (or if) it addresses the DisposableStack concern? As I understand it, the following should be equivalent:

{
  using _ = getResource();
  // ...
}

vs.

{
  using s = new DisposableStack();
  s.use(getResource());
  // ...
}

But if we're tracking function.using then the second one would break.

Fortunately, the AsyncContext champions have pivoted away from this request and are instead leaning on just throwing an error to identify bad usage, per Ron's earlier suggestion in #1 (comment)

@dead-claudia
Copy link

@dead-claudia That's an interesting and clever idea, but I'm unclear on how (or if) it addresses the DisposableStack concern? As I understand it, the following should be equivalent:

{
  using _ = getResource();
  // ...
}

vs.

{
  using s = new DisposableStack();
  s.use(getResource());
  // ...
}

But if we're tracking function.using then the second one would break.

What about something like using <value> with <stack> returning <value>, where it's also asserted that <stack> is a DisposableStack constructed with using? Not specifying specific keywords here, just general solution shape.

using s = new DisposableStack()
using getResource() with s
// ...

(I meant to include constructor calls and related super calls as well as function and method calls in the list of calls where function.using is set. Sorry for the oversight.)

Fortunately, the AsyncContext champions have pivoted away from this request and are instead leaning on just throwing an error to identify bad usage, per Ron's earlier suggestion in #1 (comment)

I, in developing a similar (non-public) system, also considered using scopes, to enable delimited React Effects-like synchronous mutation, but I ultimately tossed it due to concerns around spooky action at a distance. Instead, I just came up with something like this API:

AsyncContext.with([
    var1.set("foo"),
    var2.set("bar"),
    // ...
], () => {
    // ...
})

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants