Skip to content

Conversation

dterror-zz
Copy link

Controls whether exceeded chunk should raise an exception (and break the chain) or log and ignore
the exception. As discussed in #296

Controls whether exceeded_chunk should raise an
exception (and break the chain) or log and ignore
the exception.
@sonots
Copy link
Member

sonots commented Apr 23, 2014

This code will enqueue at lib/fluent/buffer.rb#L207-L209 even if queue size ecceeds buffer_queue_limit. It should not be occured.

(even if using @ignore_exceeded_chunk)
@dterror-zz
Copy link
Author

@sonots added a test case

@kzk
Copy link
Member

kzk commented Apr 25, 2014

-1 for this change. Here're two reasons:

First one is a philosophical thing. Fluentd's importance is 'simplicity'. The more we have options, the more beginners need to learn.

Second, you still have a limit on disk volume anyway. And if we have a disk full error while writing to the buffer, we possibly couldn't recover the data. What we want is an application level handling here.

Is just having a larger value for buffer_queue_limit would satisfy your use case?

@dterror-zz
Copy link
Author

@kzk yeah, for individual cases, increasing buffer_queue_limit is the way to go. This motivation for this feature is when a plugin is used in a chain and we want to make sure that failures in this particular plugin won't cause the chain to be broken.
Most of the discussion is in #296 and there's another pr specifically for out_copy #303.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Feature request or improve operations
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants