Skip to content

refactor: using async_openai::types::Logprobs #1625

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 24, 2025
Merged

refactor: using async_openai::types::Logprobs #1625

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
db47bc5
refactor: using async_openai::types::Logprobs
paulhendricks Jun 24, 2025
799c610
Merge branch 'main' into phendricks/refactor-logprobs
paulhendricks Jun 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 1 addition & 13 deletions lib/llm/src/protocols/openai/completions.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,6 @@
// See the License for the specific language governing permissions and
// limitations under the License.

use std::collections::HashMap;

use derive_builder::Builder;
use dynamo_runtime::protocols::annotated::AnnotationsProvider;
use serde::{Deserialize, Serialize};
Expand Down Expand Up @@ -92,7 +90,7 @@ pub struct CompletionChoice {

#[serde(skip_serializing_if = "Option::is_none")]
#[builder(default, setter(strip_option))]
pub logprobs: Option<LogprobResult>,
pub logprobs: Option<async_openai::types::Logprobs>,
}

impl ContentProvider for CompletionChoice {
Expand All @@ -107,16 +105,6 @@ impl CompletionChoice {
}
}

// TODO: validate this is the correct format
/// Legacy OpenAI LogprobResult component
#[derive(Clone, Debug, Deserialize, Serialize)]
pub struct LogprobResult {
pub tokens: Vec<String>,
pub token_logprobs: Vec<f32>,
pub top_logprobs: Vec<HashMap<String, f32>>,
pub text_offset: Vec<i32>,
}

pub fn prompt_to_string(prompt: &async_openai::types::Prompt) -> String {
match prompt {
async_openai::types::Prompt::String(s) => s.clone(),
Expand Down
4 changes: 2 additions & 2 deletions lib/llm/src/protocols/openai/completions/aggregator.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ use std::{collections::HashMap, str::FromStr};
use anyhow::Result;
use futures::StreamExt;

use super::{CompletionChoice, CompletionResponse, LogprobResult};
use super::{CompletionChoice, CompletionResponse};
use crate::protocols::{
codec::{Message, SseCodecError},
common::FinishReason,
Expand All @@ -40,7 +40,7 @@ struct DeltaChoice {
index: u64,
text: String,
finish_reason: Option<FinishReason>,
logprobs: Option<LogprobResult>,
logprobs: Option<async_openai::types::Logprobs>,
}

impl Default for DeltaAggregator {
Expand Down
Loading