Skip to content

Use nix-store --export/import instead of nix-copy-closure for build-on-target case #63

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed

Use nix-store --export/import instead of nix-copy-closure for build-on-target case #63

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4fc8d2e
nix-store --export/--import instead of nix-copy-closure
exarkun Dec 9, 2021
992c862
just use ssh compression for simplicity
exarkun Dec 10, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion deploy_nixos/nixos-deploy.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,10 +58,21 @@ log() {
echo "--- $*" >&2
}

# Use nix-copy-closure to copy some things to the target host.
# nix-copy-closure avoids copying store objects that are already on the target
# host. It performs well when copying large objects.
copyToTarget() {
NIX_SSHOPTS="${sshOpts[*]}" nix-copy-closure --to "$targetHost" "$@"
}

# Use nix-store --export/--import via ssh to copy some things to the target
# host. This removes a lot of round-trips compared to copyToTarget which is
# beneficial when copying many small store objects - such as .drv files.
exportToTarget() {
nix-store --export $(nix-store --query --requisites "$1") |
ssh -o Compression=yes "${sshOpts[@]}" "$targetHost" 'nix-store --import'
}

# assumes that passwordless sudo is enabled on the server
targetHostCmd() {
# ${*@Q} escapes the arguments losslessly into space-separted quoted strings.
Expand Down Expand Up @@ -101,7 +112,9 @@ if [[ "${buildOnTarget:-false}" == true ]]; then

# Upload derivation
log "uploading derivations"
copyToTarget "$drvPath" --gzip --use-substitutes
# Since derivations are small and there are likely many, use exportToTarget
# which should be faster than copyToTarget most of the time.
exportToTarget "$drvPath"

# Build remotely
log "building on target"
Expand All @@ -116,6 +129,9 @@ else

# Upload build results
log "uploading build results"
# Since we are copying build outputs they are probably not all very small
# objects so use copyToTarget which might be able to avoid copying some of
# them.
copyToTarget "$outPath" --gzip --use-substitutes

fi
Expand Down