flb_http_do: Errors in flb_http_get_response_data are ignored

[prashantv]

Bug Report

flb_http_do is incorrectly returning 0 even if flb_http_get_response_data fails.

Debugging the kubernetes filter, I enabeld debug logs and saw

[filter:kubernetes:kubernetes.3] Request (ns=<redacted>, pod=<redacted>) http_do=0, HTTP Status: 200

I thought that the http_do=0 would mean success, but the filter was not working correctly for the pod.

I then noticed other logs for the http_client:

[http_client] cannot increase buffer: current=32000 requested=64768 max=32000

Putting these together, I realized that the size of the response for my pod was too large. The confusing part was that the debug log suggested this response was successful (return 0), when the response was not read.

To Reproduce

Call flb_http_do to a URL that returns a large payload (E.g., ~70kb).

Expected behavior

flb_http_do to return an error, not success, when the response body is not read fully.

Code

fluent-bit/src/flb_http_client.c

Lines 1542 to 1546 in 945d81a

	ret = flb_http_get_response_data(c, 0);
	}
	/* Check 'Connection' response header */
	ret = check_connection(c);

If flb_http_get_response_data returns an error, it should be handled before the ret is overwritten with check_connection.

[edsiper]

Fixes:

• master: http_client: add response and read idle timeouts #10801
• 4.0: http_client: add response and read idle timeouts (backport v4.0) #10811