feat: bring your own llm

[jgoux]

This PR adds the ability to connect an external Ollama or OpenAI compatible model.

We use a service worker to proxy the Chat's requests to the provided external API.

Demo:

byo-llm.mp4

Remaining tasks:

• Investigate the finishReason: error we are getting from the service worker
• Integrate the service worker bundling into the build pipeline (prebuild)
• Use the specific Ollama adapter when Ollama is detected through the baseUrl Removed since we no longer recommend local models (as of today). Also Ollama just released streaming tool calls, so the OpenAI adapter should suffice.
• Add the ability to disable the usage of the external model provider without deleting its data
• Remove the need for login when using the external model provider
• Bonus: Add the ability for the user to provide their own system prompt
• Show the "set external model" sidebar button as "disabled" if the model provider is disabled
• Test with various models which support tool calling (hosted through together.ai or openrouter, local through Ollama)

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
database-build	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Nov 28, 2024 11:22pm