forked from fluent/fluent-logger-golang
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit 561c3e3
Properly stop logger during (re)connect failure
PR fluent#77 introduced a new parameter named ForceStopAsyncSend. It can be
used to tell the logger to not try to send all the log messages in its
buffer before closing. Without this parameter, the logger hangs out
whenever it has logs to write and the target Fluentd server is down.
But this new parameter is not enough: the connection is currently lazily
initialized when the logger receive its first message. This blocks the
select reading messages from the queue (until the connection is ready).
Moreover, the connection dialing uses an exponential back-off retry.
Because of that, the logger won't look for messages on `stopRunning`
channel (the channel introduced by PR fluent#77), either because it's
blocked by the Sleep used for the retry or because the connection
dialing is waiting until dialing timeout (eg. TCP timeout).
To fix these edge cases, the time.Sleep() used for back-off retry has
been transformed into a time.After(). Moreover, the dialer.Dial() call
used to connect to Fluentd has been replaced with dialer.DialContext()
and a cancellable context is now used to stop the call to that method.
The associated cancel function is stored in Fluent and got called by
Close() when ForceStopAsyncSend option is enabled.
Finally, the Fluent.run() method has been adapted to wait for both new
messages on f.pending and stop signal sent to f.stopRunning channel.
Previously, both channels were selected independently, in a procedural
fashion.
This fix is motivated by the issue described in: moby/moby#40063.
Signed-off-by: Albin Kerouanton <[email protected]>1 parent a32a5f4 commit 561c3e3Copy full SHA for 561c3e3
2 files changed
+247
-122
lines changed
0 commit comments