Add secondary Nix-based build system

[nightkr]

This won't be used for prod builds (for now), but allows Nix to cache library builds, which gives us much faster feedback loops during development (for operators that need to run inside of the K8s cluster).

This is based on (and obsoletes) the Nix build system from secret-op and listener-op.

This requires some minor changes to the operator, such as the following for Airflow:

diff --git a/rust/operator-binary/build.rs b/rust/operator-binary/build.rs
index c1b491b..a2564dd 100644
--- a/rust/operator-binary/build.rs
+++ b/rust/operator-binary/build.rs
@@ -1,10 +1,18 @@
-use stackable_airflow_crd::airflowdb::AirflowDB;
-use stackable_airflow_crd::AirflowCluster;
+use stackable_airflow_crd::{airflowdb::AirflowDB, AirflowCluster};
 use stackable_operator::crd::CustomResourceExt;
+use std::path::PathBuf;
 
 fn main() {
-    built::write_built_file().expect("Failed to acquire build-time information");
+    let out_dir = PathBuf::from(std::env::var("OUT_DIR").expect("OUT_DIR is required"));
+    built::write_built_file_with_opts(
+        // built's env module depends on a whole bunch of variables that crate2nix doesn't provide
+        // so we grab the specific env variables that we care about out ourselves instead.
+        built::Options::default().set_env(false),
+        "Cargo.toml".as_ref(),
+        &out_dir.join("built.rs"),
+    )
+    .unwrap();
 
-    AirflowCluster::write_yaml_schema("../../deploy/crd/airflowcluster.crd.yaml").unwrap();
-    AirflowDB::write_yaml_schema("../../deploy/crd/airflowdb.crd.yaml").unwrap();
+    // AirflowCluster::write_yaml_schema("../../deploy/crd/airflowcluster.crd.yaml").unwrap();
+    // AirflowDB::write_yaml_schema("../../deploy/crd/airflowdb.crd.yaml").unwrap();
 }
diff --git a/rust/operator-binary/src/main.rs b/rust/operator-binary/src/main.rs
index 654a92c..2cd1dd5 100644
--- a/rust/operator-binary/src/main.rs
+++ b/rust/operator-binary/src/main.rs
@@ -6,7 +6,7 @@ mod util;
 
 use crate::airflow_controller::AIRFLOW_CONTROLLER_NAME;
 
-use clap::Parser;
+use clap::{crate_description, crate_version, Parser};
 use futures::StreamExt;
 use stackable_airflow_crd::{
     airflowdb::{AirflowDB, AIRFLOW_DB_CONTROLLER_NAME},
@@ -32,10 +32,11 @@ use std::sync::Arc;
 
 mod built_info {
     include!(concat!(env!("OUT_DIR"), "/built.rs"));
+    pub const TARGET: Option<&str> = option_env!("TARGET");
 }
 
 #[derive(Parser)]
-#[clap(about = built_info::PKG_DESCRIPTION, author = stackable_operator::cli::AUTHOR)]
+#[clap(about, author)]
 struct Opts {
     #[clap(subcommand)]
     cmd: Command,
@@ -61,10 +62,10 @@ async fn main() -> anyhow::Result<()> {
                 tracing_target,
             );
             stackable_operator::utils::print_startup_string(
-                built_info::PKG_DESCRIPTION,
-                built_info::PKG_VERSION,
+                crate_description!(),
+                crate_version!(),
                 built_info::GIT_VERSION,
-                built_info::TARGET,
+                built_info::TARGET.unwrap_or("unknown target"),
                 built_info::BUILT_TIME_UTC,
                 built_info::RUSTC_VERSION,
             );

[nightkr]

I have tested a bit against darwin/macOS and the basic principles still work (using a remote Linux builder). However, there are some surrounding papercuts that need to be resolved for that setup to be usable (for example, we need to configure the target platform, streamLayeredImage produces a Linux-only script that won't run on darwin, and crate2nix doesn't build native darwin binaries correctly because of some dependency issues).

[soenkeliebau]

addressed comments @nightkr

[nightkr]

LGTM now. I can't approve since it was my PR originally.

[soenkeliebau]

I can :)

[nightkr]

Want the honours of merging/starting the rollout? :P

[soenkeliebau]

I only did some drudge work here, its your brainchild :)

[soenkeliebau]

Should we try and get stackabletech/hdfs-operator#312 in as well, to give people something to test with?