feat(ai): Add method to send function responses in a live session #9272
+141
−19
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
🦋 Changeset detectedLatest commit: e182238 The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Changeset File Check ✅
|
hsubox76
reviewed
Sep 23, 2025
Size Report 1Affected Products
Test Logs |
Size Analysis Report 1Affected Products
Test Logs |
paulb777
reviewed
Sep 23, 2025
hsubox76
approved these changes
Sep 24, 2025
.changeset/lazy-donuts-agree.md
Outdated
|
|
||
| Added a `sendFunctionResponses` method to `LiveSession`, allowing function responses to be sent during realtime sessions. | ||
| Fixes an issue where function responses during audio conversations caused the WebSocket connection to close. See [GitHub Issue #9264](https://github.com/firebase/firebase-js-sdk/issues/9264). | ||
| - **Breaking Change**: `StartAudioConversationOptions`' `functionCallingHandler` must now return a `Promise<FunctionResponse>`. |
There was a problem hiding this comment.
Suggested change
| - **Breaking Change**: `StartAudioConversationOptions`' `functionCallingHandler` must now return a `Promise<FunctionResponse>`. | |
| - **Breaking Change**: Changed `StartAudioConversationOptions` and `functionCallingHandler` so that they must now return a `Promise<FunctionResponse>`. |
There was a problem hiding this comment.
functionCallingHandler is what changed, and it is a property of StartAudioConversationOptions.
Updated this to be:
- **Breaking Change**: Changed the `functionCallingHandler` property in `StartAudioConversationOptions` so that it now must return a `Promise<FunctionResponse>`.
This breaking change is allowed in a minor release since the Live API is in Public Preview.
wdyt?
rachelsaunders
approved these changes
Sep 29, 2025
andrewbrook
added a commit
that referenced
this pull request
Sep 29, 2025
* docs: Fix docgen script to clean out package temp folders first (#9250) * Version Packages (#9258) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Expose setUserProperties from internal analytics instance (#9263) * feat(ai): add support for URL context (#9254) * docs(ai): tag code execution with "beta" tag (public preview) (#9274) * fix(ai): Handle empty parts when streaming (#9262) * feat(ai): Add method to send function responses in a live session (#9272) * Update ai model name used for e2e tests (#9282) --------- Co-authored-by: Google Open Source Bot <[email protected]> Co-authored-by: Christina Holland <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Daniel La Rocque <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds
LiveSession.sendFunctionResponses, which sends function responses to the server in the realtime message format. This message format is required by the Google AI backend to enable function calling during a realtime audio conversation.Since we used the non-realtime message format to send function call responses returned from
StartAudioConversationOptions.functionCallingHandler, function calling during realtime audio conversations caused the WebSocket connection to close when using thestartAudioConversationhappy path.Breaking Change:
StartAudioConversationOptions.functionCallingHandlernow must return aPromise<FunctionResponse>, instead of aPromise<Part>. I made this change because thePartneeded to be aFunctionResponsePartfor this feature to work, andFunctionResponseis the only property inFunctionResponsePart. So this just simplifies things by removing the additional layers of types. This breaking change does not break our semantic versioning policy since this API is in Public Preview.This fixes #9264