Skip to content

Cleanup uses of OpenAIMixin, simplify inference adapters #3517

@mattf

Description

@mattf

🤔 What is the technical debt you think should be addressed?

Provider chat completions completions embedding models: register models: listing models: availability models: refresh models: allowed provider-data: key config: key config: url health
anthropic mixin mixin mixin mixin mixin mixin default mixin mixin ANTHROPIC_API_KEY n/a no
azure openai mixin mixin mixin mixin mixin mixin default mixin mixin AZURE_API_KEY AZURE_API_BASE, /openai/v1 added in provider no
cerebras mixin mixin disabled mixin mixin mixin default mixin no CEREBRAS_API_KEY coded in config, /v1 added in provider no
databricks mixin disabled mixin mixin mixin (not async) mixin false mixin no DATABRICKS_TOKEN DATABRICKS_HOST, extended as needed in provider no
fireworks mixin mixin mixin mixin mixin mixin default mixin mixin FIREWORKS_API_KEY coded in config and provider no
gemini mixin mixin mixin mixin mixin mixin default mixin mixin GEMINI_API_KEY coded in provider no
groq mixin mixin mixin mixin mixin mixin default mixin mixin GROQ_API_KEY coded in config, /openai/v1 added in provider no
llama api mixin disabled disabled mixin mixin mixin default mixin mixin LLAMA_API_KEY coded in config no
nvidia mixin mixin custom (for input_type) mixin mixin mixin default mixin no NVIDIA_API_KEY NVIDIA_BASE_URL, /v1 added in provider no
ollama mixin mixin mixin custom (for :latest) mixin mixin config mixin no n/a OLLAMA_URL w/ default, /v1 added yes
openai mixin mixin mixin mixin mixin mixin default mixin mixin OPENAI_API_KEY OPENAI_BASE_URL no
runpod custom (for stream usage) mixin mixin mixin mixin mixin default mixin no RUNPOD_API_TOKEN RUNPOD_URL (no default, no mods) no
sambanova mixin mixin mixin mixin mixin mixin default mixin mixin SAMBANOVA_API_KEY coded in config no
tgi mixin mixin disabled mixin mixin mixin default mixin no n/a TGI_URL no
together mixin mixin custom (for api incompatibility) mixin mixin mixin true mixin mixin TOGETHER_API_KEY coded in config no
vertexai mixin mixin mixin mixin mixin mixin default mixin mixin GOOGLE_APPLICATION_CREDENTIALS provider constructed from VERTEX_AI_PROJECT & VERTEX_AI_LOCATION no
vllm custom (for tool_choice) mixin mixin mixin mixin mixin config mixin mixin VLLM_API_TOKEN w/ fake default VLLM_URL yes
watsonx - #3674 litellm litellm litellm litellm custom custom (always True) default no litellm WATSONX_API_KEY WATSONX_BASE_URL w/ default, also WATSONX_PROJECT_ID no

TODO: bedrock, hf::serverless, hf::endpoints

💡 What is the benefit of addressing this technical debt?

maintainability.

Other thoughts

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions