proposal: x/sync: once functions that can be retried on error

[skabbes]

sync.Once provides a great tool for initializing singletons, or performing initialization lazily. However, it is completely unusable for initializing resource who's initialization can fail (and that failure is temporary).

I am proposing a type and/or method to be added to the sync package that can handle initialization functions that can fail, to simply retry them later - the next time they are needed.

// TryDo calls `f` successively (serialized) until f first returns without error
// If `f` panics, the initialization is treated as if an error occurred
// and `TryDo` will be called again on the next invocation.
func (x *XX) TryDo(f func () error) error {
    // implementation details
    return nil
}

Simple Use cases

1. An http.Handler which lazily reads a template from over the network once

var once sync.XX
var tmpl []byte
handler := func (w http.ResponseWriter, r* http.Request) {
    err := once.TryDo(func () error {
        tmpl, err = readTemplateFromFS()
        return err
    })

    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
    }
    // do something with tmpl
}

2. Dynamically generating a lookup table on demand the first time
3. A cache library which only allocates "room" on the storage server (storage namespace / table / etc) the first time it is used.

Workaround

1. Instead of returning an error the once function would infinitely retry with a backoff policy. This allowed erroring function to fit into the sync.Once. If retries are exhausted, this is not a suitable workaround.
2. Use sync.Do but on failure, swap the sync.Once with a new sync.Once so it can retry.
3. Write your own TryDo, requiring a lock, atomic variable (to make it low overhead) and a clever defer function to deal with panics. A hand rolled version will likely not handle panics, or atomic var optimization.

Advanced use case
You are writing a library API which requires lazy initialization of some expensive reusable resource. The API looks like this.

// Search will search for matches of `q` inside book. If `ctx` is canceled, all
// search operations with Search will return with `ErrCanceled`
func (b *Book) Search(ctx context.Context, query string) ([]SearchResult, error)

However, to perform a search the first time, we would like to lazily build a search index of that book, an expensive operation, which can fail. You would like this index build operation to backoff and retry up until the limits of the context passed.

func (b *Book) Search(ctx context.Context, query string) ([]SearchResult, error) {
    err := b.once.TryDo(func () error {
        // build the search index respects the deadlines of ctx and will return an error otherwise
        return retryTilDealineOrSuccess(ctx, func () error {
            return b.buildSearchIndex(ctx)
        })
    })
    if err != nil {
        return nil, err
    }
    // ... snip ...
}

As a final comment, TryDo is more generic than Do, you can trivially build Do with TryDo but not vice versa. I would like to solicit feedback on this API addition, and if agreed I can build a design doc or submit the proposed code to implement this.

[rsc]

If retry has to happen, why can't it happen inside the function that once.Do calls?
Or maybe you are looking for golang.org/x/sync/singleflight?

/cc @bcmills @bradfitz @dvyukov

[skabbes]

It could, if infinite retry was desirable. However, if you want to use a once.Do inside a function that needs to respect a context, you need a way to jump out of the once.Do. I believe this is exactly why once.DoChan exists, so you can select against a ctx.Done().

But if the caller's context has been cancelled, there is no reason to really keep trying the initialization. Not to mention, since this initialization is implied to be expensive, you don't want to be doing it again until you need it next.

Essentially, I want this behavior to be easy to build with once.Do:

1. Try to initialize a resource with once.Do up to the limits of a context
2. Once that context has expired, give up the initialization (and fail the caller)
3. The next invocation of Do should retry the initialization
4. Repeat until success

re: singleflight - I suppose if there was a singleflightcache or something, so that subsequent (non-concurrent) calls didn't redo the initialization. That would work. It would feel odd though to use a map with a single key. I think at that point I'd just write this TryDo as library code.

[skabbes]

Oh, there is no DoChan I was mistaken, that is in singleflight. I suppose that just strengthens my point.

[robpike]

The semantics of sync.Once are very easy to understand, to use, and to trust. Adding retries is a great complication for all three. While not dismissing the problem you have, I believe the solution should be solved elsewhere, most likely in an external package, and not in the core library.

[skabbes]

Fair enough, thank you, that is the sort of decision I was looking for.

@bradfitz is this possibly something x/sync would be interested in? Feel to me pretty generally useful, and I'd hate to release it under my name to get 1 star and 0 downloads.

[dvyukov]

If initialization is expensive and/or puts load on external systems and currently fails, you probably don't want to retry it with max possible frequency. This means that the primitive also needs lastInitTime and lastError and be configurable by a backoff policy. The search index initialization probably needs to be done in a separate goroutine and start ahead of time because if it does not fit into request slot it will never be finished.

[skabbes]

I agree, I was trying to provide a more specific example to take something as abstract as "once.Do need to be able to respect context deadlines" and bring it back into reality a bit. I want to make clear that I am not asking about a particular problem that I have, rather simply noticed this limitation of sync.Once.

http context deadlines is the most familiar cancelation to gophers, but you also have queue lease expiration, user initiated cancelation, best-effort time sensitive calculations (do this before Y time, otherwise I don't care), etc etc. I was just trying to motivate the more general problem.

[dvyukov]

I mean that if we consider this problem in sufficient generality, it does not boil down to TryDo(f func () error).

[skabbes]

Yes, I believe it does. You would simply compose the once with whatever other behavior you desired including exp backoff and retry. The key feature of TryDo is you can sort of reset itself so its f can run again later.

That f can do anything: it can wait on an already launched goroutine (optimistic search index building), it can do exponential backoff and retry, or it can fail, and wait to attempt to reinitialize again later.

I don't imagine TryDo to be a complete solution, only to provide the "do this thing once, and if it fails, do it again later" behavior to compose with other behaviors (like backoff / retry / etc).

I might be missing something, do you have an example it might not cover?

[dvyukov]

@skabbes fair enough

[bcmills]

if the caller's context has been cancelled, there is no reason to really keep trying the initialization.

That's not obvious to me anyway. If the thing in question really is one-time initialization, then restarting it for each new call could be very expensive compared to just doing it once and getting it over with.

For example, consider what happens if you are connecting to a backend server, establishing the connection (for whatever reason) takes 300ms, actually using the connection takes 10ms, and each incoming call that wants to use the connection has an aggressive 30ms deadline.

With the behavior you're proposing, every operation fails in perpetuity.

With naive use of sync.Once and context.Background() (or a suitable alternative as described in #19643), all operations fail for the first 300ms or so, but then they all succeed.

since this initialization is implied to be expensive, you don't want to be doing it again until you need it next.

Don't confuse latency with throughput. Often, the action of a sync.Once has high latency but consumes very little of the program's throughput.

[bcmills]

1. Instead of returning an error the once function would infinitely retry with a backoff policy. […] If retries are exhausted, this is not a suitable workaround.

What does it mean to “exhaust” retries, or when would such exhaustion be desirable? With appropriate backoff parameters, you generally do not need to impose any artificial limit on the number of attempts: you can just keep retrying until the system comes back up.