Skip to content

[firebase_ai]: None of the documentation for the live api work?? Are we in a simulation #17413

@colbymaloy

Description

@colbymaloy

Is there an existing issue for this?

  • I have searched the existing issues.

Which plugins are affected?

Other

Which platforms are affected?

No response

Description

The docs dont contain a working example ? most of the method names dont exist. theres no example of passing audio stream into the model? What the hell is going on here guys?

late LiveModelSession _session; //should be LiveSession
final _audioRecorder = YourAudioRecorder();



// Initialize the Vertex AI Gemini API backend service
// Create a `LiveModel` instance with the model that supports the Live API
final model = FirebaseAI.vertexAI().liveModel( //should be liveGenerationModel
  model: 'gemini-2.0-flash-live-preview-04-09',
   // Configure the model to respond with audio
   config: LiveGenerationConfig(responseModalities: [ResponseModality.audio]), //should be ResponseModalities
);

_session = await model.connect();

final audioRecordStream = _audioRecorder.startRecordingStream();
// Map the Uint8List stream to InlineDataPart stream
final mediaChunkStream = audioRecordStream.map((data) {
  return InlineDataPart('audio/pcm', data);
});
await _session.startMediaStream(mediaChunkStream); //no method called startMediaStreamExists

// In a separate thread, receive the audio response from the model
await for (final message in _session.receive()) {
   // Process the received message 

///HOW??? HOW DO WE PROCESS THE MESSAGE??????
}```

What 

### Reproducing the issue

doc links https://firebase.google.com/docs/ai-logic/live-api#audio-in-audio-out

### Firebase Core version

3.13.0

### Flutter Version

3.32

### Relevant Log Output

```shell

Flutter dependencies

Expand Flutter dependencies snippet
Replace this line with the contents of your `flutter pub deps -- --style=compact`.

Additional context and comments

No response

Activity

added
type: documentationImprovements or additions to documentation
plugin: ailabel issues for firebase_ai plugin
and removed
type: bugSomething isn't working
on Jun 6, 2025
SelaseKay

SelaseKay commented on Jun 6, 2025

@SelaseKay
Contributor

Hi @colbymaloy, could you link me the documentation you're referencing? In the meantime, you can have a look at this example which captures how you pass audio to a model.

added
blocked: customer-responseWaiting for customer response, e.g. more information was requested.
and removed
Needs AttentionThis issue needs maintainer attention.
on Jun 6, 2025
AliKales

AliKales commented on Jun 6, 2025

@AliKales

Hi @SelaseKay , the example you shared is something different from what we want to achieve.

https://firebase.google.com/docs/ai-logic/live-api#audio-in-audio-out

we want to make realtime talk with Gemini, not recording audio and sending it as Content

colbymaloy

colbymaloy commented on Jun 7, 2025

@colbymaloy
Author

example

yes here is the link to the page which references methods that dont exist: https://firebase.google.com/docs/ai-logic/live-api#audio-in-audio-out

added
Needs AttentionThis issue needs maintainer attention.
and removed
blocked: customer-responseWaiting for customer response, e.g. more information was requested.
on Jun 7, 2025
SelaseKay

SelaseKay commented on Jun 9, 2025

@SelaseKay
Contributor

@kevinthecheung Could we review and possibly update this documentation in light of the concerns raised above? It appears some of the referenced methods do not exist.

willsmanley

willsmanley commented on Jun 11, 2025

@willsmanley

yep struggling a bit here as well without examples. its technically working for me but it seems to be just speaking gibberish back to me (switching languages, many back-to-back AI speech with no user speech, etc)

ilham-asgarli

ilham-asgarli commented on Jun 11, 2025

@ilham-asgarli

I was able to solve all the issues and ended up writing a blog post about it on Medium. You can take a look there if you’re interested.

17 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Needs AttentionThis issue needs maintainer attention.plugin: ailabel issues for firebase_ai plugintype: documentationImprovements or additions to documentation

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @chrisn-au@google-oss-bot@colbymaloy@willsmanley@SelaseKay

        Issue actions

          [firebase_ai]: None of the documentation for the live api work?? Are we in a simulation · Issue #17413 · firebase/flutterfire