feat(embedder): make embedder support openai compatible model like qwen #169
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hello @XingLiu0923, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
Summary of Changes
Hello team, gemini-code-assist here to provide a summary of this pull request. This PR introduces support for using embedding models that are compatible with the OpenAI API standard. This allows users to configure DeepWiki to use self-hosted or third-party services that expose an OpenAI-like embedding endpoint, such as Alibaba Qwen, by setting environment variables and updating the embedder.json configuration.
Highlights
- OpenAI-Compatible Embedder Support: The core feature added is the ability to use embedding models that are compatible with the OpenAI API, expanding the range of usable models beyond just OpenAI's official offerings.
- Flexible Configuration via Environment Variables: Configuration files (
.json) can now contain placeholders like${ENV_VAR}which will be automatically substituted with the corresponding environment variable values when the configuration is loaded. This is used to configure the OpenAI-compatible embedder's API key and base URL. - Centralized Embedder Initialization: The logic for initializing the
adal.Embedderhas been moved into a new helper functionget_embedderinapi/tools/embedder.py. This function handles selecting the correct configuration (default/OpenAI compatible or Ollama) and passing initialization arguments (initialize_kwargs) if they exist in the config. - Updated Documentation: The main README and its translated versions have been updated to include a new section explaining how to configure and use OpenAI-compatible embedding models.
- Example Configuration Provided: A new example configuration file (
api/config/embedder_openai_compatible.json.bak) is included to demonstrate how to set up an OpenAI-compatible embedder using environment variable placeholders.
Changelog
Click here to see the changelog
- README.es.md
- Added a new section (lines 242-254) explaining how to use OpenAI-compatible embedding models, including steps for replacing the config and setting environment variables.
- Added a blank line at the end of the file (line 334).
- README.ja.md
- Added a blank line at the end of the file (line 447).
- README.kr.md
- Added a blank line at the end of the file (line 430).
- README.md
- Added a new section (lines 244-256) explaining how to use OpenAI-compatible embedding models, including steps for replacing the config and setting environment variables.
- README.vi.md
- Added a blank line at the end of the file (line 423).
- README.zh.md
- Added a new section (lines 356-368) explaining how to use OpenAI-compatible embedding models, including steps for replacing the config and setting environment variables.
- api/config.py
- Imported the
remodule (line 4). - Added a new recursive function
replace_env_placeholders(lines 52-67) to find and replace${ENV_VAR}patterns in string values within dictionaries and lists using environment variables. - Modified
load_json_config(lines 86-87) to callreplace_env_placeholderson the loaded configuration dictionary before returning it.
- Imported the
- api/config/embedder.json.bak
- Added this file (lines 1-25) as a backup of the default
embedder.jsonconfiguration.
- Added this file (lines 1-25) as a backup of the default
- api/config/embedder_openai_compatible.json.bak
- Added this file (lines 1-29) as an example configuration for using an OpenAI-compatible embedder, demonstrating the use of
initialize_kwargswith${OPENAI_API_KEY}and${OPENAI_API_BASE_URL}placeholders.
- Added this file (lines 1-29) as an example configuration for using an OpenAI-compatible embedder, demonstrating the use of
- api/data_pipeline.py
- Imported the new
get_embedderfunction fromapi.tools.embedder(line 18). - Replaced the direct initialization of
adal.Embedder(lines 369-376) with a call to the newget_embedderfunction (line 371).
- Imported the new
- api/rag.py
- Imported the new
get_embedderfunction fromapi.tools.embedder(line 9). - Replaced the direct initialization of
self.embedder(lines 231-240) with a call to the newget_embedderfunction (line 232).
- Imported the new
- api/tools/embedder.py
- Added this new file (lines 1-22) containing the
get_embedderfunction, which encapsulates the logic for creating theadal.Embedderinstance based on the configuration and handlinginitialize_kwargs.
- Added this new file (lines 1-22) containing the
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
An API so grand,
Compatible across the land.
Embeddings flow free,
For all the world to see,
With config in your hand.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Code Review
This pull request introduces support for OpenAI-compatible embedding models, enhancing the flexibility of the application. The use of environment variables for configuration is a good practice. The changes to api/tools/embedder.py and the updates to the README files are also positive.
Summary of Findings
- Bug in Nested List Placeholder Replacement: The
replace_env_placeholdersfunction inapi/config.pydoes not correctly replace placeholders in lists that are nested within dictionaries. This can lead to configurations not being fully processed as expected. - Handling of Unresolved Placeholders: When an environment variable for a placeholder is not found, the placeholder string remains in the configuration. While this is one way to handle it, it can lead to downstream type errors or unexpected behavior. Logging a warning in such cases would improve debuggability.
Merge Readiness
The pull request introduces a valuable feature and is generally well-implemented. However, there is a high-severity bug in the replace_env_placeholders function concerning the processing of lists nested within dictionaries, and a medium-severity enhancement opportunity for logging unresolved placeholders. I recommend addressing these issues, particularly the bug, before merging to ensure the feature's robustness. I am not authorized to approve pull requests, so please ensure further review and approval from authorized maintainers after addressing the feedback.
|
@sng-asyncfunc sorry for bothering you, but I appriciate if you could look into this. Thank you. |
|
Thanks for contributing to this! This is HUGE! |
…en (AsyncFuncAI#169) * feat(embedder): make embedder support openai compatible model like qwen * refactor config py --------- Co-authored-by: Xing Liu <[email protected]>
Support for OpenAI-Compatible Embedder Models (e.g., Qwen)
This PR introduces support for embedding models that adhere to the OpenAI API compatibility standard. This allows deepwiki-open to leverage a wider range of self-hosted or commercially available models beyond just OpenAI's official offerings for embedding tasks.
Key Changes
typethat allows for specifying OpenAI-compatible API endpoints.api/config/embedder_openai_compatible.json.bak, is included to demonstrate how to set up an OpenAI-compatible embedder, using Qwen as an example.Benefits
How to Use
To utilize an OpenAI-compatible embedder, you'll need to:
Set Environment Variables:
OPENAI_API_BASE_URL: Set this environment variable to the base URL of your OpenAI-compatible API endpoint (e.g.,http://localhost:8000/v1).OPENAI_API_KEY: If your API requires an API key, set this environment variable accordingly. Keep in mind that some self-hosted solutions might not require this.Configure
embedder.json:api/config/embedder_openai_compatible.json.bak.embedder.jsonfile, you'll need to specify theinitialize_kwargsvariable. This variable allows you to pass additional initialization arguments to the embedder, such as the specific model name you wish to use (e.g.,{"model": "Qwen/Qwen-7B-Chat"}).Local Testing Confirmation
Local testing has been successfully completed and confirmed to be working as expected.
This enhancement significantly broadens the capabilities of deepwiki-open. Let me know if you have any questions or feedback!