Feature: Support self-hosted gitlab instance

[Axel-Schneider]

Improve url management for including self-hosted gitlab server.

Issue #26

[gemini-code-assist]

Hello @Axel-Schneider, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

Summary of Changes

This pull request improves the application's ability to interact with self-hosted GitLab instances. It introduces a new type parameter to several functions, allowing users to specify the repository type (GitHub, GitLab, or Bitbucket). This enables the application to correctly format URLs and API requests for different Git platforms, including handling access tokens for private repositories. The changes also include refactoring to consolidate token handling and URL parsing, making the code more maintainable and robust.

Highlights

• GitLab Support: Adds support for self-hosted GitLab instances by introducing a type parameter to handle different URL formats and authentication methods.
• Token Handling: Consolidates token handling by using a single token parameter instead of separate parameters for GitHub, GitLab, and Bitbucket tokens.
• URL Parsing: Improves URL parsing for GitLab repositories, including handling both cloud-hosted and self-hosted instances.
• Code Refactoring: Refactors code to improve readability and maintainability, such as removing redundant checks and consolidating API request logic.

Changelog

Click here to see the changelog

• api/data_pipeline.py
  • • Added urllib.parse import for URL parsing.
  • • Modified download_repo function to accept a type parameter to handle different repository types.
  • • Updated download_repo to correctly format clone URLs for GitHub, GitLab (including self-hosted), and Bitbucket using access tokens.
  • • Refactored get_gitlab_file_content to handle both cloud and self-hosted GitLab instances, improving URL parsing and API request construction.
  • • Removed retry logic for 'master' branch in get_gitlab_file_content to simplify error handling.
  • • Modified get_file_content to accept a type parameter to determine which repository content retrieval function to use.
  • • Updated DatabaseManager class methods (prepare_database, _create_repo, prepare_retriever) to accept a type parameter for repository handling.
• api/rag.py
  • • Modified prepare_retriever function to accept a type parameter for repository handling.
• api/simple_chat.py
  • • Consolidated token parameters into a single token parameter in ChatCompletionRequest.
  • • Modified chat_completions_stream to use the type parameter to determine the repository type and pass the token to prepare_retriever and get_file_content.
• src/app/[owner]/[repo]/page.tsx
  • • Removed separate token parameters (github_token, gitlab_token, bitbucket_token) and replaced them with a single token parameter.
  • • Added repoUrl to RepoInfo type.
  • • Modified addTokensToRequestBody to use the single token parameter.
  • • Updated fetchRepositoryStructure to use extractUrlPath and extractUrlDomain for GitLab URL parsing and to fetch the file tree and README content.
  • • Modified API calls to include the type parameter in the request body.
  • • Updated getRepoUrl to use the RepoInfo object and return the repoUrl if available.
• src/app/page.tsx
  • • Added extractUrlPath import for URL parsing.
  • • Added a regex for custom git urls.
  • • Modified the URL parsing logic to handle custom git urls.
  • • Modified the URL parameters to use a single token.
• src/components/Ask.tsx
  • • Modified AskProps to use RepoInfo object instead of separate token and URL parameters.
  • • Updated API calls to use the RepoInfo object and pass the token in the request body.
• src/types/repoinfo.tsx
  • • Added a new RepoInfo type to encapsulate repository information, including owner, repo, type, token, localPath, and repoUrl.
• src/types/wiki/wikipage.tsx
  • • Added new fields for hierarchy
• src/types/wiki/wikisection.tsx
  • • Added new file for wiki sections
• src/types/wiki/wikistructure.tsx
  • • Added new file for wiki structure
• src/utils/getRepoUrl.tsx
  • • Created a new utility function getRepoUrl to generate repository URLs based on the RepoInfo object.
• src/utils/urlDecoder.tsx
  • • Created a new utility function extractUrlDomain and extractUrlPath to extract the domain and path from a URL.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

---

A self-hosted server's gleam,
GitLab's instance, a coder's dream.
With tokens secured,
The code is assured,
A seamless workflow, it would seem.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces support for self-hosted GitLab instances, improving URL management for various Git repository types. The changes involve adding a 'type' parameter to several functions to handle different repository URLs and authentication methods. Overall, the changes seem well-structured and address the issue effectively.

Summary of Findings

• Error Handling in get_gitlab_file_content: The get_gitlab_file_content function simplifies error handling by raising a ValueError for any failure. While this makes the code cleaner, it might be too generic and could obscure specific error details that could be useful for debugging.
• Redundant Checks: The original code had checks for specific domains (github.com, gitlab.com, bitbucket.org) in multiple places. The changes replace these with a 'type' parameter, which is a good improvement. However, ensure that the 'type' parameter is validated to prevent unexpected behavior.
• Code Duplication: There are some instances of duplicated code blocks, such as the token handling logic. Consider refactoring these into a separate function to improve maintainability.

Merge Readiness

The pull request is almost ready for merging. However, I recommend addressing the error handling in get_gitlab_file_content and validating the 'type' parameter to ensure robustness. Also, consider refactoring duplicated code blocks. I am unable to approve this pull request, and recommend that others review and approve this code before merging. At a minimum, the high severity issues should be addressed before merging.