Nix support #371
Nix support #371
Conversation
WalkthroughA comprehensive Nix-based deployment and development setup for SourceBot was introduced. This includes a Nix flake, NixOS module, overlay, and system/test configurations, along with a detailed deployment guide update. New files define systemd services, package builds, virtual machine setup, and a hash manifest for dependencies, enabling reproducible builds and deployments. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant NixOS System
participant SourceBot Service
participant PostgreSQL
participant Redis
User->>NixOS System: Deploy using Nix flake/module
NixOS System->>SourceBot Service: Start systemd units (web, backend, zoekt)
SourceBot Service->>PostgreSQL: Connect/setup database
SourceBot Service->>Redis: Connect/setup cache
SourceBot Service->>User: Expose API/Web on configured port
Poem
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 8
🔭 Outside diff range comments (1)
nix/missing-hashes.json (1)
1-181: Improve automated hash verificationThe current suggestion uses
dist.shasum(SHA-1) from the npm registry, which won’t match the SHA-256 entries innix/missing-hashes.json. Instead, fetch each package tarball and compute its SHA-256. For example:#!/usr/bin/env bash # Verify SHA-256 for a sample of packages against missing-hashes.json # Sample list of entries <name>@<version> packages=( "@esbuild/[email protected]" "@next/[email protected]" ) for pkg in "${packages[@]}"; do name=${pkg%@*} version=${pkg#*@} # Fetch tarball URL url=$(curl -s "https://registry.npmjs.org/$name/$version" | jq -r '.dist.tarball') echo "Verifying $pkg from $url" # Compute SHA-256 (Nix’s prefetch-url is ideal here) actual=$(nix-prefetch-url --unpack --type sha256 "$url" 2>/dev/null) expected=$(jq -r --arg key "${name}@npm:${version}" '.[$key]' nix/missing-hashes.json) echo " expected: $expected" echo " actual: $actual" if [[ "$actual" == "$expected" ]]; then echo " ✅ OK" else echo " ❌ MISMATCH" exit 1 fi done• No changes needed in
nix/missing-hashes.jsonitself.
• Incorporate this (or similar) check into your CI to catch stale or incorrect hashes automatically.
🧹 Nitpick comments (7)
nix/overlay.nix (1)
1-12: Well-structured overlay with appropriate package additions.The overlay correctly adds the sourcebot package and updates zoekt to a specific revision. The structure follows Nix overlay best practices.
Consider adding comments to clarify the purpose of the zoekt override:
final: prev: { sourcebot = final.callPackage ./sourcebot.nix {}; + # Override zoekt with updated version compatible with sourcebot zoekt = prev.zoekt.overrideAttrs (old: rec { vendorHash = "sha256-laiBp+nMWEGofu7zOgfM2b8MIC+Dfw7eCLgb/5zf9oo="; src = final.fetchFromGitHub { owner = "sourcegraph"; repo = "zoekt"; + # Updated to revision with sourcebot compatibility improvements rev = "12a2f4ad075359a09bd8a91793acb002211217aa"; hash = "sha256-JByTgJsnqLlP7hNbQumM4zqZZuj7igc2V35vw0ahCqM="; }; }); }Note: The static analysis warning about "sourcegraph-access-token" is a false positive - this is just the repository owner name in the GitHub URL.
docs/docs/deployment-guide.mdx (1)
101-208: Comprehensive NixOS deployment documentation with good examples.The documentation provides clear step-by-step instructions for deploying SourceBot on NixOS, including proper secret management and configuration examples.
Consider these improvements for better security and usability:
- Reduce concurrency values in the example - The current values (1000) are extremely high:
settings = { resyncConnectionIntervalMs = 1000 * 60 * 60 * 24 * 7; # 1 week reindexIntervalMs = 1000 * 60 * 60 * 24 * 7; # 1 week - maxRepoIndexingJobConcurrency = 1000; # 8 default - maxConnectionSyncJobConcurrency = 1000; # 8 default - maxRepoGarbageCollectionJobConcurrency = 1000; # 8 default + maxRepoIndexingJobConcurrency = 8; # Default value + maxConnectionSyncJobConcurrency = 8; # Default value + maxRepoGarbageCollectionJobConcurrency = 8; # Default value };
- Add security note about file permissions:
+ <Warning> + Ensure your secrets files have restrictive permissions (600) and are owned by the sourcebot user. + </Warning>
- Add firewall configuration guidance:
+ <Step title="Configure firewall (optional)"> + If you need to access SourceBot from external networks, configure your firewall: + + ```nix + networking.firewall.allowedTCPPorts = [ 7734 ]; # Default SourceBot port + ``` + </Step>nix/nixosTest.nix (1)
1-31: Good basic test structure for NixOS module validation.The test correctly validates that all essential SourceBot services start up and respond to HTTP requests.
Consider enhancing the test for better coverage and robustness:
testScript = '' start_all() machine.wait_for_unit("sourcebot-web.service") machine.wait_for_unit("sourcebot-backend.service") machine.wait_for_unit("sourcebot-db-setup.service") machine.wait_for_unit("postgresql.service") machine.wait_for_unit("sourcebot-zoekt.service") - machine.succeed("${pkgs.curl}/bin/curl http://localhost:7734") + + # Wait for web service to be ready + machine.wait_for_open_port(7734) + + # Test web interface responds correctly + machine.succeed("${pkgs.curl}/bin/curl -f http://localhost:7734") + + # Verify database connection + machine.succeed("systemctl is-active postgresql.service") + + # Check logs for any startup errors + machine.succeed("! journalctl -u sourcebot-web.service --since=-1m | grep -i error") '';Also consider using a minimal but valid configuration instead of empty JSON:
configPath = "${pkgs.writeText "config" '' - { - } + { + "$schema": "https://github.com/raw/sourcebot-dev/sourcebot/main/schemas/v3/index.json", + "connections": {} + } ''}"nix/microvm.nix (1)
40-43: Consider security implications of dev environment settings.The empty root password and autologin are appropriate for development but should be documented as such.
+ # Development environment settings - NOT for production users.users.root.password = ""; # Enable autologin for root user services.getty.autologinUser = "root";nix/nixosModule.nix (2)
52-52: Fix typo in option description- description = "Open Firwall ports for SourceBot"; + description = "Open Firewall ports for SourceBot";
64-68: Consider security implications of default hostnameThe default hostname
0.0.0.0binds to all network interfaces, which might expose the service more broadly than intended. Consider using127.0.0.1as default for better security.flake.nix (1)
15-15: Consider supporting additional architecturesThe flake is currently limited to
x86_64-linux. Consider supporting additional architectures likeaarch64-linuxfor broader compatibility.- flake-utils.lib.eachSystemPassThrough ["x86_64-linux"] (system: { + flake-utils.lib.eachSystemPassThrough ["x86_64-linux" "aarch64-linux"] (system: {and
- // flake-utils.lib.eachSystem ["x86_64-linux"] ( + // flake-utils.lib.eachSystem ["x86_64-linux" "aarch64-linux"] (Also applies to: 30-30
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
flake.lockis excluded by!**/*.lock
📒 Files selected for processing (8)
docs/docs/deployment-guide.mdx(3 hunks)flake.nix(1 hunks)nix/microvm.nix(1 hunks)nix/missing-hashes.json(1 hunks)nix/nixosModule.nix(1 hunks)nix/nixosTest.nix(1 hunks)nix/overlay.nix(1 hunks)nix/sourcebot.nix(1 hunks)
🧰 Additional context used
🪛 Gitleaks (8.26.0)
nix/overlay.nix
8-8: Sourcegraph is a code search and navigation engine.
(sourcegraph-access-token)
🔇 Additional comments (2)
nix/microvm.nix (1)
45-62: Well-configured SourceBot service for development.The service configuration is appropriate for development with debug logging and a comprehensive test configuration including both GitHub and the JSON schema reference.
nix/nixosModule.nix (1)
191-191: Verify different restart policies for servicesThe web service uses
Restart = "always"while the backend service usesRestart = "on-failure". Is this intentional?Also applies to: 224-224
| volumes = [ | ||
| { | ||
| mountPoint = "/var"; | ||
| image = "sourcebot-var.img"; | ||
| size = 1 * 1024; # 10GB | ||
| } | ||
| ]; |
There was a problem hiding this comment.
Fix inconsistent volume size configuration.
The comment indicates 10GB but the actual size is configured as 1GB (1 * 1024 MB).
volumes = [
{
mountPoint = "/var";
image = "sourcebot-var.img";
- size = 1 * 1024; # 10GB
+ size = 10 * 1024; # 10GB
}
];Or if 1GB is intended:
- size = 1 * 1024; # 10GB
+ size = 1 * 1024; # 1GB🤖 Prompt for AI Agents
In nix/microvm.nix between lines 29 and 35, the volume size is set to 1 * 1024
MB but the comment incorrectly states 10GB. To fix this inconsistency, either
update the size to 10 * 1024 MB to match the 10GB comment or change the comment
to reflect the actual 1GB size if that is intended.
| { | ||
| type = "user"; | ||
| id = "sourecbot"; | ||
| mac = "02:00:00:00:00:10"; | ||
| } |
There was a problem hiding this comment.
Fix typo in interface identifier.
There's a typo in the interface ID that should be corrected.
interfaces = [
{
type = "user";
- id = "sourecbot";
+ id = "sourcebot";
mac = "02:00:00:00:00:10";
}
];📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| { | |
| type = "user"; | |
| id = "sourecbot"; | |
| mac = "02:00:00:00:00:10"; | |
| } | |
| interfaces = [ | |
| { | |
| type = "user"; | |
| id = "sourcebot"; | |
| mac = "02:00:00:00:00:10"; | |
| } | |
| ]; |
🤖 Prompt for AI Agents
In nix/microvm.nix around lines 8 to 12, the interface ID "sourecbot" contains a
typo. Correct the ID by changing "sourecbot" to the intended correct identifier,
likely "sourcebot" or the appropriate name as per your naming conventions.
| runHook preInstall | ||
|
|
||
| cp -r packages/web/.next/standalone $out | ||
| cp -r node_modules/* $out/node_modules |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Remove redundant node_modules copying
The node_modules directory is being copied multiple times which could cause conflicts and is inefficient. Consider consolidating these operations.
- cp -r node_modules/* $out/node_modules
-
mkdir -p $out/packages/web
cp -r packages/web/public $out/packages/web/public
mkdir -p $out/packages/web/.next
cp -r packages/web/.next/static $out/packages/web/.next/static
mkdir -p $out/packages/backend
- cp -r packages/backend/node_modules/* $out/node_modules
cp -r packages/backend/* $out/packages/backend
cp -r packages/db/* $out/packages/db
cp -r packages/schemas/* $out/packages/schemas
cp -r packages/crypto/* $out/packages/crypto
cp -r packages/error $out/packages/error
cp -r packages/mcp $out/packages/mcp
cp -r node_modules/* $out/node_modulesAlso applies to: 57-57, 65-65
🤖 Prompt for AI Agents
In nix/sourcebot.nix at lines 49, 57, and 65, the node_modules directory is
copied multiple times redundantly. Remove the duplicate cp -r node_modules/*
$out/node_modules commands at lines 57 and 65, keeping only one copy operation
to avoid conflicts and improve efficiency.
|
|
||
| mkdir -p $out/bin | ||
|
|
||
| rm -rf $out/pacakages/web/.next/cache |
There was a problem hiding this comment.
Fix typo in directory path
There's a typo in the path: pacakages should be packages.
Apply this fix:
- rm -rf $out/pacakages/web/.next/cache
+ rm -rf $out/packages/web/.next/cache📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| rm -rf $out/pacakages/web/.next/cache | |
| rm -rf $out/packages/web/.next/cache |
🤖 Prompt for AI Agents
In nix/sourcebot.nix at line 69, correct the typo in the directory path by
changing "pacakages" to "packages" in the rm command to ensure the correct cache
directory is targeted for removal.
| ++ optional (cfg.envFile == null) [ | ||
| "AUTH_SECRET=00000000000000000000000000000000000000000000" | ||
| "SOURCEBOT_ENCRYPTION_KEY=00000000000000000000000000000000" | ||
| ]; |
There was a problem hiding this comment.
Avoid hardcoded secrets in environment
Using hardcoded placeholder secrets even as defaults is a security risk. Consider requiring users to provide an envFile or generate random secrets.
Consider either:
- Making
envFilerequired when auth is enabled - Generating random secrets at runtime
- Providing clear documentation that these are insecure defaults
Would you like me to implement a solution that generates secure random secrets?
Also applies to: 218-221
🤖 Prompt for AI Agents
In nix/nixosModule.nix around lines 185 to 188 and also lines 218 to 221, the
code uses hardcoded placeholder secrets for AUTH_SECRET and
SOURCEBOT_ENCRYPTION_KEY, which is a security risk. To fix this, remove the
hardcoded secrets and instead require users to provide an envFile when
authentication is enabled or implement logic to generate secure random secrets
at runtime. Additionally, update documentation to clearly state that default
secrets are insecure if you keep any defaults.
| package = lib.mkOption { | ||
| type = types.package; | ||
| default = self.packages.${pkgs.system}.sourcebot; | ||
| description = "Package to use for nixos-cli"; |
There was a problem hiding this comment.
Fix incorrect package description
The description mentions "nixos-cli" but should reference "sourcebot".
- description = "Package to use for nixos-cli";
+ description = "Package to use for sourcebot";📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| description = "Package to use for nixos-cli"; | |
| description = "Package to use for sourcebot"; |
🤖 Prompt for AI Agents
In nix/nixosModule.nix at line 20, the package description incorrectly
references "nixos-cli" instead of "sourcebot". Update the description string to
correctly mention "sourcebot" to accurately reflect the package purpose.
| configPath = mkOption { | ||
| type = types.str; | ||
| description = "Path to the SourceBot configuration file"; | ||
| }; |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Add default value for configPath
The configPath option lacks a default value, which could cause runtime errors if not explicitly set by users.
Consider adding a sensible default:
configPath = mkOption {
type = types.str;
+ default = "${cfg.dataDir}/config.json";
description = "Path to the SourceBot configuration file";
};📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| configPath = mkOption { | |
| type = types.str; | |
| description = "Path to the SourceBot configuration file"; | |
| }; | |
| configPath = mkOption { | |
| type = types.str; | |
| default = "${cfg.dataDir}/config.json"; | |
| description = "Path to the SourceBot configuration file"; | |
| }; |
🤖 Prompt for AI Agents
In nix/nixosModule.nix around lines 55 to 58, the configPath option is missing a
default value, which may lead to runtime errors if users do not set it. Add a
default value field to the mkOption definition, providing a sensible default
path string for the SourceBot configuration file to ensure safe fallback
behavior.
| ExecStart = "${pkgs.prisma}/bin/prisma migrate deploy --schema ${cfg.package}/packages/db/prisma/schema.prisma"; | ||
| Environment = [ | ||
| "DATABASE_URL=postgresql://sourcebot@localhost:${toString config.services.postgresql.settings.port}/sourcebot" | ||
| "PATH=${makeBinPath (with pkgs; [prisma openssl])}" |
There was a problem hiding this comment.
Import makeBinPath function
The makeBinPath function is used but not imported.
- "PATH=${makeBinPath (with pkgs; [prisma openssl])}"
+ "PATH=${lib.makeBinPath (with pkgs; [prisma openssl])}"📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| "PATH=${makeBinPath (with pkgs; [prisma openssl])}" | |
| "PATH=${lib.makeBinPath (with pkgs; [prisma openssl])}" |
🤖 Prompt for AI Agents
In nix/nixosModule.nix at line 154, the function makeBinPath is used but not
imported. To fix this, import makeBinPath from the appropriate package or module
at the beginning of the file so it is available for use in constructing the PATH
variable.
|
@msukkari , if you are intrested in this change, I would fix coderabbitai suggestions. But also I recommend to somehow run Nix in CI. Basically these commands:
Sometimes you would need to update the Another option is to push this change to nixpkgs main repository, then any NixOS user would be able to deploy Sourcebot, but you would have less control over its deployment options and it depends on contributors (or you) to maintain its version. |
There was a problem hiding this comment.
Thanks for this change @abraverm! Just left a few comments on the docs page to fix broken links and styling
Adding the build to CI shouldn't be too hard and is something we can look into once we get some bandwidth. I think we should be safe to merge this for now and add CI once usage of this deployment flow increases?
| ]; | ||
| } | ||
| ``` | ||
| [Learn more about NixOS flakes](/docs/installation/nixos-flakes). |
There was a problem hiding this comment.
is this meant to link to external docs? It seems this is broken since no nixos-flakes docs page exists
| </Step> | ||
| <Step title="Enable Sourcebot"> | ||
| The following NixOS configuration will enable Sourcebot and set it up to run with the provided configuration. | ||
| Additional options could be found in the [source file](../../nix/nixosModule.nix) |
There was a problem hiding this comment.
It seems this is meant to link to the nixosModule.nix file in the repo, in which case you have to link to the file itself on GitHub. Mintlify (the docs library we're using) doesn't support linking to files outside of the docs folder like this as far as I'm aware
|
|
||
| - [sops-nix](https://github.com/Mic92/sops-nix) example: | ||
|
|
||
| ```nix |
There was a problem hiding this comment.
nit: please add additional params to the code snippets to follow the styling of the rest of the document
For example:
bash wrap icon="terminal" Create example config
The icon can be any valid lucide icon: https://lucide.dev/icons/
For more info check out the mintlify docs: https://mintlify.com/docs/code#code-block-options
| })}"; | ||
| }; | ||
| ``` | ||
| </Step> |
There was a problem hiding this comment.
nit: please add the Login and Done steps from the container deployment here. Copy and paste should be fine!
|
Also don't feel like you need to address all of the coderabbit comments! It sometimes provides reviews that aren't really needed |
Package Sourcebot with Nix, NixOS module for deployment, integration test and microvm.
Summary by CodeRabbit
New Features
Documentation