Add support for stream usage in Azure OpenAi #2858
Conversation
andresssantos
force-pushed
the
azure-openai-stream-usage
branch
from
0c8ce47 to
8fa1dbc
Compare
| } | ||
|
|
||
| public void setStreamUsage(Boolean enableStreamUsage) { | ||
| this.streamOptions = (enableStreamUsage) ? new ChatCompletionStreamOptions().setIncludeUsage(true) : null; |
There was a problem hiding this comment.
We need an explicit property to set the enableStreamUsage value.
There was a problem hiding this comment.
This explicit property needs to drive how the usage is included in AzureOpenAiChatModel. Please check OpenAIChatModel StreamOptions#includeUsage for reference.
There was a problem hiding this comment.
Thank you for your review and feedback, @ilayaperumalg!
Based on your comments, I have made the following changes:
- Introduced an explicit property (
enableStreamUsage) inAzureOpenAiChatOptions. - Updated the setter so it no longer modifies
streamOptionsdirectly. - Adjusted the logic in
AzureOpenAiChatModelto use theenableStreamUsageproperty to determine whether to include usage information in streaming responses, following the pattern ofOpenAiChatModel.
Could you please check if the current implementation meets the expectations?
Thanks again for your guidance!
Signed-off-by: Andres da Silva Santos <[email protected]>
Signed-off-by: Andres da Silva Santos <[email protected]>
Signed-off-by: Andres da Silva Santos <[email protected]>
andresssantos
force-pushed
the
azure-openai-stream-usage
branch
from
ffe2bed to
9b81f11
Compare
|
Hi @ilayaperumalg, this looks good to me as well. |
|
Thank you @andresssantos for the follow up with the PR and @mkheck for the review. We'll get this reviewed and merged soon. |
|
Rebased, squashed and merged as 7bb553e @andresssantos Thanks very much for the contribution! |
Description
Related issue: spring-ai#1724
This PR introduces support for streamUsage in the
AzureOpenAiChatOptionsclass. Additionally:AzureOpenAiChatOptionsTeststo reflect the changes.azure-openai-chat.adoc.