[Feature] OpenAI-Compatible Tools API + Streaming for Hermes & Mistral models #5649

K-Mistele · 2024-06-18T18:34:00Z

DRAFT: OpenAI Tool Use Checklist

This (Draft) PR will add support for OpenAI-style tool calling in a way that is minimally opinionated about tool use formats & prompt formatting.

The following features are expected to be supported:

Custom tool use system prompt template (if desired) -- prevent opinionation about if/how the model uses a system prompt to enable tool use
Custom tool call return value prompt template (if desired) -- prevent opinionation about the format that tool return values should be passed to the model
Support for tool_choice="auto" - named tool choice is already supported via guided decoding
Streaming tool call responses from the chat completions API
Verified Support & examples for at least the following models:
- Mistral 7B Instruct v0.3
- Nous Research's Hermes 2 Pro Models function-calling models including Hermes 2 Pro Llama 3 8B

I'd welcome anyone who wants to contribute on this, and would be happy to add you to the Constellate AI vllm fork that this PR is based off of - please just leave a comment!

Checklist/roadmap:

FIX #3237 #4656 (link existing issues this PR will resolve)

BEFORE SUBMITTING, PLEASE READ THE CHECKLIST BELOW AND FILL IN THE DESCRIPTION ABOVE

PR Checklist (Click to Expand)

Thank you for your contribution to vLLM! Before submitting the pull request, please ensure the PR meets the following criteria. This helps vLLM maintain the code quality and improve the efficiency of the review process.

PR Title and Classification

Only specific types of PRs will be reviewed. The PR title is prefixed appropriately to indicate the type of change. Please use one of the following:

[Bugfix] for bug fixes.
[CI/Build] for build or continuous integration improvements.
[Doc] for documentation fixes and improvements.
[Model] for adding a new model or improving an existing model. Model name should appear in the title.
[Frontend] For changes on the vLLM frontend (e.g., OpenAI API server, LLM class, etc.)
[Kernel] for changes affecting CUDA kernels or other compute kernels.
[Core] for changes in the core vLLM logic (e.g., LLMEngine, AsyncLLMEngine, Scheduler, etc.)
[Hardware][Vendor] for hardware-specific changes. Vendor name should appear in the prefix (e.g., [Hardware][AMD]).
[Misc] for PRs that do not fit the above categories. Please use this sparingly.

Note: If the PR spans more than one category, please include all relevant prefixes.

Code Quality

The PR need to meet the following code quality standards:

We adhere to Google Python style guide and Google C++ style guide.
Pass all linter checks. Please use format.sh to format your code.
The code need to be well-documented to ensure future contributors can easily understand the code.
Include sufficient tests to ensure the project to stay correct and robust. This includes both unit tests and integration tests.
Please add documentation to docs/source/ if the PR modifies the user-facing behaviors of vLLM. It helps vLLM user understand and utilize the new features or changes.

Notes for Large Changes

Please keep the changes as concise as possible. For major architectural changes (>500 LOC excluding kernel/data/config/test), we would expect a GitHub issue (RFC) discussing the technical design and justification. Otherwise, we will tag it with rfc-required and might not go through the PR.

What to Expect for the Reviews

The goal of the vLLM team is to be a transparent reviewing machine. We would like to make the review process transparent and efficient and make sure no contributor feel confused or frustrated. However, the vLLM team is small, so we need to prioritize some PRs over others. Here is what you can expect from the review process:

After the PR is submitted, the PR will be assigned to a reviewer. Every reviewer will pick up the PRs based on their expertise and availability.
After the PR is assigned, the reviewer will provide status update every 2-3 days. If the PR is not reviewed within 7 days, please feel free to ping the reviewer or the vLLM team.
After the review, the reviewer will put an action-required label on the PR if there are changes required. The contributor should address the comments and ping the reviewer to re-review the PR.
Please respond to all comments within a reasonable time frame. If a comment isn't clear or you disagree with a suggestion, feel free to ask for clarification or discuss the suggestion.

Thank You

Finally, thank you for taking the time to read these guidelines and for your interest in contributing to vLLM. Your contributions make vLLM a great tool for everyone!

…nja template

…= "auto"; set default `tool_choice` to "auto"

… instead of vllm/engine/arg_utils.py

…none" AND NOT "auto"

… to receive the arguments & validate them

…long with validation

…requests

…to-tool-choice and --tool-use-prompt-template are specified

K-Mistele · 2024-06-19T02:19:25Z

Progress! I as of current commits, I can now get the hermes 2 pro model to generate a tool call using the --enable-auto-tool-choice and --tool-use-prompt-template flags:

Server:

python -m vllm.entrypoints.openai.api_server --model NousResearch/Hermes-2-Pro-Llama-3-8B --tool-use-prompt-template examples/tool_template_hermes_2_pro.jinja --enable-api-tools --enable-auto-tool-choice

Client:

python examples/openai_chat_completion_client_with_tools.py

Result

Chat completion results:
ChatCompletion(id='cmpl-1354f3f373574d7aa0e1bf0b78915188', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='<tool_call>{"arguments": {"city": "Dallas", "state": "TX", "unit": "fahrenheit"}, "name": "get_current_weather"}</tool_call>', role='assistant', function_call=None, tool_calls=[]), stop_reason=None)], created=1718763539, model='NousResearch/Hermes-2-Pro-Llama-3-8B', object='chat.completion', system_fingerprint=None, usage=CompletionUsage(completion_tokens=33, prompt_tokens=367, total_tokens=400))

Now, working on getting it to work for non-streaming responses - then, streaming!

K-Mistele · 2024-06-19T02:36:51Z

A question I asked in the discord, with some open questions about how to handle configuration:

Setting up function calling models for an open model requires a lot of configurations if you want to be unopinionated about the model. Here is a brief list of all the parameters that would be needed:

enable "auto" tool choice - allow models to choose the function to call (supported for some models) or ONLY tool_choice="" for named tool choice via guided decoding
tool use prompt template - how do you render the list of tools provided in the request into the prompt / system prompt for the conversation?
tool use prompt role - what is the role of that message? defaults to system.
tool use response start token - different models (e.g. Hermes 2 Pro models vs. Mistral 7B Instruct v0.3) use different tokens in their tokenizer to indicate the start of a tool call response vs. a chat response. it's important that this is configured correctly so that we can know whether to send the model's response as a chat response or tool response, and how to stream the response if stream=True. Because there does not seem to be a specific convention for defining this in a uniform way across model tokenizers, it will be necessary for the user to inform the API server which token indicates a tool response is starting, in order to have fully OpenAI API-compatible tool responses.
tool result response/return value message template the template for returning tool results to the model so it can generated based on results of the tool. - e.g. <tool_response>{"name": "function_name_here", "content": TOOL_RETURN_VALUE_HERE}</tool_response> for Hermes 2 pro models
tool result response/return value message role - the ROLE to use for that message, e.g. tool for Hermes 2 pro models

The question is, it is better to have all of these as separate CLI flags, or would a JSON configuration file be preferable so that people can create (and track in version control!) configs that work for popular models?)

K-Mistele · 2024-06-19T03:25:29Z

Please see ongoing conversation with the Hugging Face team, Nous Research & transformers maintainer here - this will make it MUCH easier to implement OpenAI API-compatible tool calling into vLLM regardless of model prompt/tokenizer configs.

HF PR for Hermes 2 Pro: https://huggingface.co/NousResearch/Hermes-2-Pro-Llama-3-8B/discussions/13#66724ea9bd5875ad665f1416

HF PR for Mistral 7B instruct v0.3: https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3/discussions/35

Once these are merged, there will be a STANDARD way in transformers to handle templating in tool responses just like for templating chat conversations into prompts with a chat template, and hopefully to pull out tool calls from generated text.

interstellarninja · 2024-06-20T05:41:24Z

'<tool_call>{"arguments": {"city": "Dallas", "state": "TX", "unit": "fahrenheit"}, "name": "get_current_weather"}</tool_call>'

hey great initiative and nice to see Hermes Pro model's tool calls working.

there's a slight issue with this tool call -- our format requires new lines after <tool_call> XML tags:

<tool_call>
{"arguments": {"city": "Dallas", "state": "TX", "unit": "fahrenheit"}, "name": "get_current_weather"}
</tool_call>

Also tool choice should also work since it's basically passing the chosen tool only as part of

K-Mistele · 2024-06-20T15:12:59Z

Thanks! Tool choice is already working via guided decoding, but I will update the PR to fix the template

…ines as expected

K-Mistele · 2024-06-20T15:19:16Z

Ok the most recent commit seems to fix it:
python examples/openai_chat_completion_client_with_tools.py

ChatCompletion(id='cmpl-48f019602ab64f4ab49c3563318c6d1f', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='<tool_call>\n{"arguments": {"city": "Dallas", "state": "TX", "unit": "fahrenheit"}, "name": "get_current_weather"}\n</tool_call>', role='assistant', function_call=None, tool_calls=[]), stop_reason=None)], created=1718896666, model='NousResearch/Hermes-2-Pro-Llama-3-8B', object='chat.completion', system_fingerprint=None, usage=CompletionUsage(completion_tokens=34, prompt_tokens=368, total_tokens=402))

frankabc12341 · 2024-06-21T23:04:03Z

@K-Mistele
Hi. My vllm version is 0.5.0 post1. But, when I run the command

python -m vllm.entrypoints.openai.api_server --model /home/asus/autodl-tmp/qwen/Qwen2-7B-Instruct --tool-use-prompt-template /home/asus/autodl-tmp/examples/chatml.jinja --enable-api-tools --enable-auto-tool-choice
it shows api_server.py: error: unrecognized arguments: --tool-use-prompt-template --enable-api-tools --enable-auto-tool-choice

In addition, I can't find openai_chat_completion_client_with_tools.py in the example fille.

Can you give me some advice?

K-Mistele · 2024-06-22T20:30:53Z

@K-Mistele Hi. My vllm version is 0.5.0 post1. But, when I run the command

python -m vllm.entrypoints.openai.api_server --model /home/asus/autodl-tmp/qwen/Qwen2-7B-Instruct --tool-use-prompt-template /home/asus/autodl-tmp/examples/chatml.jinja --enable-api-tools --enable-auto-tool-choice it shows api_server.py: error: unrecognized arguments: --tool-use-prompt-template --enable-api-tools --enable-auto-tool-choice

In addition, I can't find openai_chat_completion_client_with_tools.py in the example fille.

Can you give me some advice?

Please see my reply here. This is a draft pull request, which means it has not been merged into vLLM's codebase, and it is not ready to be merged yet. It is still a work-in-progress. As such, none of its additions or features are available in vLLM yet. If you are interested in testing or contributing to this pull request, please see the origin fork at the tool-use branch. However, please be aware that the capabilities discussed in this PR are incomplete and have not been robustly tested.

aw632 · 2024-06-24T21:40:11Z

if the tool use token is called, pull out the tool call JSON from the rest of the response (should be array)

Given that guided decoding is not enabled for "auto" tool use, what is the error handling planned in case the LLM does not output valid JSON?

K-Mistele · 2024-06-24T23:05:52Z

if the tool use token is called, pull out the tool call JSON from the rest of the response (should be array)

Given that guided decoding is not enabled for "auto" tool use, what is the error handling planned in case the LLM does not output valid JSON?

Great question! Unfortunately since each model that supports tool calling uses its' own format for function calls (as opposed to tool choice with guided decoding, where we're forcing the LLM to call a specific tool in a specific format at decode-time) the response format is up to the model and its' trainer.

At this point, we are still exploring ways to handle extraction of tool calls from disparate formats to OpenAI-compatible calls in a way that isn't opinionated (we want to support multiple formats including Mistral, Hermes 2 Pro, Firefunction, etc). Until we have a good answer on this, we probably won't have a good answer for how to handle errors. There are a couple possible options once we have attempted to extract the model's tool call format into OpenAI's:

return the call to the client without validation, and allow the client to validate the tool call with libraries like Instructor, pydantic or Zod
return an HTTP error if the model generates an invalid tool call
automatically attempt to re-run the generation - perhaps only if the user specifies a specific CLI flag, since this could lead to performance issues or infinite retry loops depending on the model and other configurations like temperature, top_p and top_k

In the interim, we may try to solve this problem by "ignoring" it - detect if the model is generating a tool call, but avoid the destructuring/extraction issue by returning the tool call in the model's format to the client. Not exactly sure what this would look like, but it's a possibility.

K-Mistele · 2024-06-24T23:33:57Z

Copy/pasting from discord:

I'd really like to make progress on tool use and right now with hugging face adding support into transformers for passing tools into the chat template, that solves one of the main issues

I think the blocker now is figuring out how to handle decodiong "auto" choice tool calls from each model's specific format (mistral vs. Hermes pro vs. firefunction) and returning that to the client ESPECIALLY when streaming is requested.

Until there's a "canonical" way to do decode model-specific tool-calls into the OpenAI format e.g. through transformers or "reverse templates" or something, it might be best to approach this like chat templates which is to try & support it as good as we can where possible

Here's what I mean by this:

mistral's format is very close to openAI's format and is probably close in terms of implementation. If a tool is being called, the first token from the response indicates this, and then a JSOn array is generated. I can use a partial JSON parser to stream tool arguments etc as they are generated to the client, basically without any translation
Hermes 2 Pro's format is different and uses multiple XML tags <tool_call></tool_call> for each call with the JSON inside. I should also be able to handle some streaming here with XML extraction and regex.
Firefunction v2 uses a totally different format as well.

So each one needs a very different implementation. I propose creating a ToolCallParser abstract class that can be implemented for different models like mistral and hermes. If a user is using a tool-calling model, they can use a CLI flag to toggle which parser they want, if they want one at all. If not, a tool call would be treated like a regular chat completion, and they can handle it client-side.

This way, we can ship support for tool calls incrementally in the absence of a commonly-accepted "best practice" on how to do this. People can also add support for other models that are important to them in a minimally-invasive way. Then, once a better way to implement this more broadly is available, we can deprecate this approach

I'd really appreciate feedback on this before moving forward and would especially love to hear if @mgoin and @simon-mo would consider this an appropriate approach that would be likely to be approved.

simon-mo · 2024-06-24T23:48:16Z

@br3no would love to get your feedback and review for this PR!

TimPietrusky · 2024-09-05T00:01:46Z

I was following along the whole time, thank you so much @K-Mistele ❤️

meetzuber · 2024-09-05T12:47:18Z

How can I use it with llama 3.1 8b model?

pbasov · 2024-09-06T12:06:44Z

@meetzuber you have to write a prompt template for l3.1 if I understand the implementation correctly.

gislerro · 2024-09-06T13:39:33Z

@meetzuber you have to write a prompt template for l3.1 if I understand the implementation correctly.

From my understanding you have to write a ToolParser aswell.

I'm experimenting with a more general approach to tool parsing that generalizes over different models

Since OpenAI API compatible tool calls only require two string arguments name and arguments a regex for custom tool calling must only define two capture groups with those names, such a regex can be validated with a "meta" regex like:

(?=.*?\(\?P<name>.*?\))(?=.*?\(\?P<arguments>.*?\))

The tool calling regex must then be provided with a cli arg --tool-call-regex.

Then as an example for the Llama 3.1 model with JSON based tool calling your --tool-call-regex looks like:

{"name": "(?P<name>.*?)", "parameters": (?P<arguments>.*?)}

which would then extract the tool call: https://regex101.com/r/RhZ4zx/1

For a different custom tool calling format just provide another --tool-call-regex:

<function=(?P<name>.*?)>(?P<arguments>.*?)<\/function>

which also extracts tool call(s): https://regex101.com/r/9pM6IL/2

K-Mistele · 2024-09-06T13:58:12Z

How can I use it with llama 3.1 8b model?

I will be adding support in a separate PR

K-Mistele · 2024-09-06T13:59:44Z

@meetzuber you have to write a prompt template for l3.1 if I understand the implementation correctly.

From my understanding you have to write a ToolParser aswell.

I'm experimenting with a more general approach to tool parsing that generalizes over different models

Since OpenAI API compatible tool calls only require two string arguments name and arguments a regex for custom tool calling must only define two capture groups with those names, such a regex can be validated with a "meta" regex like:
(?=.*?\(\?P<name>.*?\))(?=.*?\(\?P<arguments>.*?\))
The tool calling regex must then be provided with a cli arg --tool-call-regex.

Then as an example for the Llama 3.1 model with JSON based tool calling your --tool-call-regex looks like:
{"name": "(?P<name>.*?)", "parameters": (?P<arguments>.*?)}
which would then extract the tool call: https://regex101.com/r/RhZ4zx/1

For a different custom tool calling format just provide another --tool-call-regex:
<function=(?P<name>.*?)>(?P<arguments>.*?)<\/function>
which also extracts tool call(s): https://regex101.com/r/9pM6IL/2

Unfortunately this approach does not work with streaming. Supporting tools in streaming mode was a core requirement of this PR since many applications use streaming mode by default if they are user-facing, and there is no way to “turn off streaming” if the model starts generating a tool call.

pbasov · 2024-09-06T15:20:03Z

@K-Mistele Documentation would be much appreciated, so the community can be quick to add support for future models with tool support.

meetzuber · 2024-09-06T17:53:11Z

Only below options are mentioned in tool/function calling. and there are only 2 options for tool call parser mistral and hermes.
There is no --tool-call-regex option there.

--enable-auto-tool-choice – mandatory Auto tool choice. tells vLLM that you want to enable the model to generate its own tool calls when it deems appropriate.

--tool-call-parser – select the tool parser to use - currently either hermes or mistral. Additional tool parsers will continue to be added in the future.

--chat-template – optional for auto tool choice. the path to the chat template which handles tool-role messages and assistant-role messages that contain previously generated tool calls. Hermes and Mistral models have tool-compatible chat templates in their tokenizer_config.json files, but you can specify a custom template. This argument can be set to tool_use if your model has a tool use-specific chat template configured in the tokenizer_config.json. In this case, it will be used per the transformers specification. More on this here from HuggingFace; and you can find an example of this in a tokenizer_config.json here

https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html#automatic-function-calling

gislerro · 2024-09-06T18:33:00Z

There is no --tool-call-regex option there.

That's not implemented on the main branch

meetzuber · 2024-09-06T18:38:13Z

Only below options are mentioned in tool/function calling. and there are only 2 options for tool call parser mistral and hermes.
There is no --tool-call-regex option there.

--enable-auto-tool-choice – mandatory Auto tool choice. tells vLLM that you want to enable the model to generate its own tool calls when it deems appropriate.

--tool-call-parser – select the tool parser to use - currently either hermes or mistral. Additional tool parsers will continue to be added in the future.

--chat-template – optional for auto tool choice. the path to the chat template which handles tool-role messages and assistant-role messages that contain previously generated tool calls. Hermes and Mistral models have tool-compatible chat templates in their tokenizer_config.json files, but you can specify a custom template. This argument can be set to tool_use if your model has a tool use-specific chat template configured in the tokenizer_config.json. In this case, it will be used per the transformers specification. More on this here from HuggingFace; and you can find an example of this in a tokenizer_config.json here

https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html#automatic-function-calling

That's not implemented on the main branch

These are implemented in v0.6.0 release.
https://github.com/vllm-project/vllm/releases/tag/v0.6.0

vipulgote1999 · 2024-09-07T04:09:10Z

@K-Mistele can mentioned which is the pull request for llama3.1 tool support...ithink most people is been waiting for it.

JackYangzg · 2024-09-09T07:48:52Z

@K-Mistele can mentioned which is the pull request for Qwen2 tool support...ithink most people is been waiting for it.

K-Mistele · 2024-09-09T15:17:14Z

@K-Mistele can mentioned which is the pull request for llama3.1 tool support...ithink most people is been waiting for it.

There is not a pull request for it yet because it's still a work-in-progress and not ready for review. I can create a draft though.

K-Mistele · 2024-09-09T15:19:20Z

@K-Mistele can mentioned which is the pull request for Qwen2 tool support...ithink most people is been waiting for it.

There is not a PR for this, and at this point I hadn't planned on it. If this is something you're interested, please feel free to create an issue for a feature request, and tag me in it. If it gets enough interest, I'll add it.

…l models (vllm-project#5649) Co-authored-by: constellate <[email protected]> Co-authored-by: Kyle Mistele <[email protected]> Signed-off-by: Alvant <[email protected]>

…l models (vllm-project#5649) Co-authored-by: constellate <[email protected]> Co-authored-by: Kyle Mistele <[email protected]> Signed-off-by: LeiWang1999 <[email protected]>

constellate and others added 10 commits June 16, 2024 01:21

add hermes 2 pro function calling template

6b762e5

feat(example): add example chat completion with tool usage

606ec64

feat: add CLI argument for OpenAI API-style tool use system prompt ji…

d27446f

…nja template

feat: add better validation for tool_choice, support for tool_choice …

bdf48a1

…= "auto"; set default `tool_choice` to "auto"

fix(cli): set OpenAI tool args in vllm/entrypoints/openai/cli_args.py…

9493c67

… instead of vllm/engine/arg_utils.py

fix(types): add "auto" as an option for tool_choice in pydantic models

35c9aa7

fix: validation - guided decoding not valid with tool_choice = auto

8ff80fb

feat: ensure guided deoding is only applied when tool_choice is NOT "…

3c4acb1

…none" AND NOT "auto"

feat(cli): update CLI args for auto tool choice and OpenAIServingChat…

9c5ef66

… to receive the arguments & validate them

feat: add loading in the system prompt jinja template if specified; a…

f1a1e7b

…long with validation

This was referenced Jun 18, 2024

[Frontend][OpenAI] Add support for OpenAI tools calling #4656

Closed

OpenAI Tools / function calling v2 #3237

Closed

K-Mistele added 3 commits June 18, 2024 19:14

wip: add a case for tool choice = auto when handling chat completion …

07a67d2

…requests

fix: hermes 2 pro prompt template to prevent newlines

590a559

feat: handle building the system prompt via template with --enable-au…

6919acf

…to-tool-choice and --tool-use-prompt-template are specified

fix: remove debugging log lines

db9c29a

fix(template): update hermes 2 pro template with newlines to get newl…

c16aa9a

…ines as expected

lk1983823 mentioned this pull request Jun 21, 2024

api_server.py: error: unrecognized arguments: --tool-use-prompt-template --enable-api-tools --enable-auto-tool-choice #5730

Closed

SolitaryThinker mentioned this pull request Sep 4, 2024

[bugfix] Upgrade minimum OpenAI version #8169

Merged

devonthomas35 mentioned this pull request Sep 4, 2024

Release v0.6.0 #8144

Closed

1 task

BeautyyuYanli mentioned this pull request Sep 5, 2024

feat: Function calling for OpenAI Compatible Chat Completion API bentoml/OpenLLM#452

Closed

virajmehta mentioned this pull request Sep 5, 2024

Investigate whether we can add tool calling support to vLLM now tensorzero/tensorzero#274

Closed

virajmehta mentioned this pull request Sep 6, 2024

Implement a thorough tool calling solution for vLLM models tensorzero/tensorzero#169

Closed

K-Mistele mentioned this pull request Sep 9, 2024

[Bug]: Mistral Large Instruct 2407 tool calling leakage #8301

Closed

1 task

tomeras91 mentioned this pull request Sep 11, 2024

[Bugfix]: Fix the logic for deciding if tool parsing is used #8366

Merged

K-Mistele mentioned this pull request Sep 12, 2024

[Frontend][Feature] support tool calling for internlm/internlm2_5-7b-chat model #8405

Merged

terrytangyuan mentioned this pull request Sep 29, 2024

Standardize discovery, installation and usage of LLM functions, agents and tools kserve/kserve#3829

Open

DarkLight1337 mentioned this pull request Nov 9, 2024

[Frontend] request.tool_choice is not completely exhausted in non-stream mode #5764

Closed

Uh oh!

[Feature] OpenAI-Compatible Tools API + Streaming for Hermes & Mistral models #5649

[Feature] OpenAI-Compatible Tools API + Streaming for Hermes & Mistral models #5649

Uh oh!

Conversation

K-Mistele commented Jun 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

DRAFT: OpenAI Tool Use Checklist

PR Title and Classification

Code Quality

Notes for Large Changes

What to Expect for the Reviews

Thank You

Uh oh!

K-Mistele commented Jun 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

K-Mistele commented Jun 19, 2024

Uh oh!

K-Mistele commented Jun 19, 2024

Uh oh!

interstellarninja commented Jun 20, 2024

Uh oh!

K-Mistele commented Jun 20, 2024

Uh oh!

K-Mistele commented Jun 20, 2024

Uh oh!

frankabc12341 commented Jun 21, 2024

Uh oh!

K-Mistele commented Jun 22, 2024

Uh oh!

aw632 commented Jun 24, 2024

Uh oh!

K-Mistele commented Jun 24, 2024

Uh oh!

K-Mistele commented Jun 24, 2024

Uh oh!

simon-mo commented Jun 24, 2024

Uh oh!

TimPietrusky commented Sep 5, 2024

Uh oh!

meetzuber commented Sep 5, 2024

Uh oh!

pbasov commented Sep 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gislerro commented Sep 6, 2024

Uh oh!

K-Mistele commented Sep 6, 2024

Uh oh!

K-Mistele commented Sep 6, 2024

Uh oh!

pbasov commented Sep 6, 2024

Uh oh!

meetzuber commented Sep 6, 2024

Uh oh!

gislerro commented Sep 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

meetzuber commented Sep 6, 2024

Uh oh!

vipulgote1999 commented Sep 7, 2024

Uh oh!

JackYangzg commented Sep 9, 2024

Uh oh!

K-Mistele commented Sep 9, 2024

Uh oh!

K-Mistele commented Sep 9, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

27 participants

K-Mistele commented Jun 18, 2024 •

edited

Loading

K-Mistele commented Jun 19, 2024 •

edited

Loading

pbasov commented Sep 6, 2024 •

edited

Loading

gislerro commented Sep 6, 2024 •

edited

Loading