Skip to content

Small updates to allow for gpt-oss generation #2049

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

iamlemec
Copy link
Contributor

@iamlemec iamlemec commented Aug 6, 2025

Compatibility updates for gpt-oss generation:

  • Seems like seq_rm doesn't accept -1 anymore, so just use seq_id = 0 for generation (is this fine for the server?)
  • The gpt-oss template uses strftime_now in the template, so provide that function

Of course, this also needs a llama.cpp version bump (not included here) for the new model architecture.

@abetlen
Copy link
Owner

abetlen commented Aug 7, 2025

Thank you @iamlemec !

@abetlen abetlen closed this Aug 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants